Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [..................................X.........X...X.X...X...X.] duration: 2.059608625s FAIL (2025-08-11 02:50:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 02:49:25.639325678 (57h23m20.070405809s) [upload=59823]: Starting upload Aug 11 02:49:25.639374762 (57h23m20.070454893s) [upload=59823]: Done starting upload Aug 11 02:49:25.639426411 (57h23m20.070506532s) [upload=59823]: Uploading segments... Aug 11 02:49:25.639461839 (57h23m20.07054197s) [upload=59823]: Got next segment. Inline: true Aug 11 02:49:25.639465265 (57h23m20.070545396s) [upload=59823]: Waiting for error group managing segments... Aug 11 02:49:25.639495032 (57h23m20.070575163s) [upload=59823]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:49:25.639498939 (57h23m20.07057907s) [upload=59823]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:49:25.639501415 (57h23m20.070581546s) [upload=59823]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:49:25.853369357 (57h23m20.284449488s) [upload=59823]: Done uploading segments... Aug 11 02:49:25.853386330 (57h23m20.284466461s) [upload=59823]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.059643882s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................X.....X.............................] duration: 14.811820243s FAIL (2025-08-11 02:50:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 14.81184514s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.14724545s FAIL (2025-08-11 02:50:17 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.147265088s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................X.....X............................] duration: 12.380872113s FAIL (2025-08-11 02:50:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 12.380901889s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................X.....X...........................] duration: 7.303697897s FAIL (2025-08-11 02:49:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 7.303734487s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [...................................X.........X...X.X...X...X] duration: 862.979854ms FAIL (2025-08-11 02:49:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 02:49:25.639325678 (57h23m20.070405809s) [upload=59823]: Starting upload Aug 11 02:49:25.639374762 (57h23m20.070454893s) [upload=59823]: Done starting upload Aug 11 02:49:25.639426411 (57h23m20.070506532s) [upload=59823]: Uploading segments... Aug 11 02:49:25.639461839 (57h23m20.07054197s) [upload=59823]: Got next segment. Inline: true Aug 11 02:49:25.639465265 (57h23m20.070545396s) [upload=59823]: Waiting for error group managing segments... Aug 11 02:49:25.639495032 (57h23m20.070575163s) [upload=59823]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:49:25.639498939 (57h23m20.07057907s) [upload=59823]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:49:25.639501415 (57h23m20.070581546s) [upload=59823]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:49:25.853369357 (57h23m20.284449488s) [upload=59823]: Done uploading segments... Aug 11 02:49:25.853386330 (57h23m20.284466461s) [upload=59823]: Upload finished. err: metaclient: Too Many Requests Duration:* 863.000263ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.32004963s FAIL (2025-08-11 02:49:22 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.320068476s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.253243424s FAIL (2025-08-11 02:49:00 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.253263372s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................X.....X..........................] duration: 11.547548986s FAIL (2025-08-11 02:48:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 11.547579485s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.174871565s FAIL (2025-08-11 02:48:29 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.174895191s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [....................................X.........X...X.X...X...] duration: 1.828463417s FAIL (2025-08-11 02:48:19 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 02:46:36.553289605 (57h20m30.984369736s) [upload=59776]: Starting upload Aug 11 02:46:36.553336444 (57h20m30.984416565s) [upload=59776]: Done starting upload Aug 11 02:46:36.553383845 (57h20m30.984463976s) [upload=59776]: Uploading segments... Aug 11 02:46:36.553423640 (57h20m30.984503771s) [upload=59776]: Got next segment. Inline: true Aug 11 02:46:36.553427238 (57h20m30.984507369s) [upload=59776]: Waiting for error group managing segments... Aug 11 02:46:36.553458838 (57h20m30.984538969s) [upload=59776]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:46:36.553463016 (57h20m30.984543147s) [upload=59776]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:46:36.553465551 (57h20m30.984545682s) [upload=59776]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:46:36.827372928 (57h20m31.258453049s) [upload=59776]: Done uploading segments... Aug 11 02:46:36.827390823 (57h20m31.258470944s) [upload=59776]: Upload finished. err: <nil> Duration:* 1.828482184s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.374011359s FAIL (2025-08-11 02:47:52 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.37403295s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.....................................X.........X...X.X...X..] duration: 1.829423393s FAIL (2025-08-11 02:47:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 02:46:36.553289605 (57h20m30.984369736s) [upload=59776]: Starting upload Aug 11 02:46:36.553336444 (57h20m30.984416565s) [upload=59776]: Done starting upload Aug 11 02:46:36.553383845 (57h20m30.984463976s) [upload=59776]: Uploading segments... Aug 11 02:46:36.553423640 (57h20m30.984503771s) [upload=59776]: Got next segment. Inline: true Aug 11 02:46:36.553427238 (57h20m30.984507369s) [upload=59776]: Waiting for error group managing segments... Aug 11 02:46:36.553458838 (57h20m30.984538969s) [upload=59776]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:46:36.553463016 (57h20m30.984543147s) [upload=59776]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:46:36.553465551 (57h20m30.984545682s) [upload=59776]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:46:36.827372928 (57h20m31.258453049s) [upload=59776]: Done uploading segments... Aug 11 02:46:36.827390823 (57h20m31.258470944s) [upload=59776]: Upload finished. err: <nil> Duration:* 1.829447009s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................X.....X.........................] duration: 7.201104997s FAIL (2025-08-11 02:47:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 7.201131338s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.066234942s FAIL (2025-08-11 02:47:14 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.066250413s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [......................................X.........X...X.X...X.] duration: 1.835605332s FAIL (2025-08-11 02:47:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 02:46:36.553289605 (57h20m30.984369736s) [upload=59776]: Starting upload Aug 11 02:46:36.553336444 (57h20m30.984416565s) [upload=59776]: Done starting upload Aug 11 02:46:36.553383845 (57h20m30.984463976s) [upload=59776]: Uploading segments... Aug 11 02:46:36.553423640 (57h20m30.984503771s) [upload=59776]: Got next segment. Inline: true Aug 11 02:46:36.553427238 (57h20m30.984507369s) [upload=59776]: Waiting for error group managing segments... Aug 11 02:46:36.553458838 (57h20m30.984538969s) [upload=59776]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:46:36.553463016 (57h20m30.984543147s) [upload=59776]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:46:36.553465551 (57h20m30.984545682s) [upload=59776]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:46:36.827372928 (57h20m31.258453049s) [upload=59776]: Done uploading segments... Aug 11 02:46:36.827390823 (57h20m31.258470944s) [upload=59776]: Upload finished. err: <nil> Duration:* 1.835640791s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.......................................X.........X...X.X...X] duration: 1.571679081s FAIL (2025-08-11 02:46:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 02:46:36.553289605 (57h20m30.984369736s) [upload=59776]: Starting upload Aug 11 02:46:36.553336444 (57h20m30.984416565s) [upload=59776]: Done starting upload Aug 11 02:46:36.553383845 (57h20m30.984463976s) [upload=59776]: Uploading segments... Aug 11 02:46:36.553423640 (57h20m30.984503771s) [upload=59776]: Got next segment. Inline: true Aug 11 02:46:36.553427238 (57h20m30.984507369s) [upload=59776]: Waiting for error group managing segments... Aug 11 02:46:36.553458838 (57h20m30.984538969s) [upload=59776]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:46:36.553463016 (57h20m30.984543147s) [upload=59776]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:46:36.553465551 (57h20m30.984545682s) [upload=59776]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:46:36.827372928 (57h20m31.258453049s) [upload=59776]: Done uploading segments... Aug 11 02:46:36.827390823 (57h20m31.258470944s) [upload=59776]: Upload finished. err: <nil> Duration:* 1.571705621s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................X.....X........................] duration: 8.348985135s FAIL (2025-08-11 02:46:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 8.349014311s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [X.......................................X.........X...X.X...] duration: 1.857087489s FAIL (2025-08-11 02:46:04 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:44:07.301050704 (57h18m1.732130835s) [upload=59733]: Starting upload Aug 11 02:44:07.301099076 (57h18m1.732179197s) [upload=59733]: Done starting upload Aug 11 02:44:07.301157980 (57h18m1.732238111s) [upload=59733]: Uploading segments... Aug 11 02:44:07.301213967 (57h18m1.732294098s) [upload=59733]: Got next segment. Inline: true Aug 11 02:44:07.301217603 (57h18m1.732297735s) [upload=59733]: Waiting for error group managing segments... Aug 11 02:44:07.301250566 (57h18m1.732330697s) [upload=59733]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:44:07.301254674 (57h18m1.732334805s) [upload=59733]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:44:07.301257069 (57h18m1.7323372s) [upload=59733]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:44:07.621151950 (57h18m2.052232091s) [upload=59733]: Done uploading segments... Aug 11 02:44:07.621174123 (57h18m2.052254254s) [upload=59733]: Upload finished. err: <nil> Duration:* 1.857119149s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.399680725s FAIL (2025-08-11 02:46:02 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.399928579s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [XX.......................................X.........X...X.X..] duration: 2.003128198s FAIL (2025-08-11 02:45:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:44:07.301050704 (57h18m1.732130835s) [upload=59733]: Starting upload Aug 11 02:44:07.301099076 (57h18m1.732179197s) [upload=59733]: Done starting upload Aug 11 02:44:07.301157980 (57h18m1.732238111s) [upload=59733]: Uploading segments... Aug 11 02:44:07.301213967 (57h18m1.732294098s) [upload=59733]: Got next segment. Inline: true Aug 11 02:44:07.301217603 (57h18m1.732297735s) [upload=59733]: Waiting for error group managing segments... Aug 11 02:44:07.301250566 (57h18m1.732330697s) [upload=59733]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:44:07.301254674 (57h18m1.732334805s) [upload=59733]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:44:07.301257069 (57h18m1.7323372s) [upload=59733]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:44:07.621151950 (57h18m2.052232091s) [upload=59733]: Done uploading segments... Aug 11 02:44:07.621174123 (57h18m2.052254254s) [upload=59733]: Upload finished. err: <nil> Duration:* 2.003157604s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................X.....X.......................] duration: 6.763949129s FAIL (2025-08-11 02:45:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 6.763975389s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.....X......................] duration: 7.548552641s FAIL (2025-08-11 02:45:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 7.548580095s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.22974803s FAIL (2025-08-11 02:45:06 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.229778629s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [.XX.......................................X.........X...X.X.] duration: 1.91619987s FAIL (2025-08-11 02:45:04 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:44:07.301050704 (57h18m1.732130835s) [upload=59733]: Starting upload Aug 11 02:44:07.301099076 (57h18m1.732179197s) [upload=59733]: Done starting upload Aug 11 02:44:07.301157980 (57h18m1.732238111s) [upload=59733]: Uploading segments... Aug 11 02:44:07.301213967 (57h18m1.732294098s) [upload=59733]: Got next segment. Inline: true Aug 11 02:44:07.301217603 (57h18m1.732297735s) [upload=59733]: Waiting for error group managing segments... Aug 11 02:44:07.301250566 (57h18m1.732330697s) [upload=59733]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:44:07.301254674 (57h18m1.732334805s) [upload=59733]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:44:07.301257069 (57h18m1.7323372s) [upload=59733]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:44:07.621151950 (57h18m2.052232091s) [upload=59733]: Done uploading segments... Aug 11 02:44:07.621174123 (57h18m2.052254254s) [upload=59733]: Upload finished. err: <nil> Duration:* 1.916237421s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.277961214s FAIL (2025-08-11 02:44:11 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.277978357s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X.....X.....................] duration: 8.962670882s FAIL (2025-08-11 02:44:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 8.962687323s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [..XX.......................................X.........X...X.X] duration: 1.415116144s FAIL (2025-08-11 02:44:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:44:07.301050704 (57h18m1.732130835s) [upload=59733]: Starting upload Aug 11 02:44:07.301099076 (57h18m1.732179197s) [upload=59733]: Done starting upload Aug 11 02:44:07.301157980 (57h18m1.732238111s) [upload=59733]: Uploading segments... Aug 11 02:44:07.301213967 (57h18m1.732294098s) [upload=59733]: Got next segment. Inline: true Aug 11 02:44:07.301217603 (57h18m1.732297735s) [upload=59733]: Waiting for error group managing segments... Aug 11 02:44:07.301250566 (57h18m1.732330697s) [upload=59733]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:44:07.301254674 (57h18m1.732334805s) [upload=59733]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:44:07.301257069 (57h18m1.7323372s) [upload=59733]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:44:07.621151950 (57h18m2.052232091s) [upload=59733]: Done uploading segments... Aug 11 02:44:07.621174123 (57h18m2.052254254s) [upload=59733]: Upload finished. err: <nil> Duration:* 1.4151355s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.881985824s FAIL (2025-08-11 02:43:32 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.882013507s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...XX.......................................X.........X...X.] duration: 1.834601131s FAIL (2025-08-11 02:43:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:42:31.914140022 (57h16m26.345220143s) [upload=59703]: Starting upload Aug 11 02:42:31.914204895 (57h16m26.345285026s) [upload=59703]: Done starting upload Aug 11 02:42:31.914293916 (57h16m26.345374047s) [upload=59703]: Uploading segments... Aug 11 02:42:31.914330556 (57h16m26.345410687s) [upload=59703]: Got next segment. Inline: true Aug 11 02:42:31.914337289 (57h16m26.34541747s) [upload=59703]: Waiting for error group managing segments... Aug 11 02:42:31.914397023 (57h16m26.345477204s) [upload=59703]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:42:31.914404247 (57h16m26.345484389s) [upload=59703]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:42:31.914408556 (57h16m26.345488697s) [upload=59703]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:42:32.193134135 (57h16m26.624214266s) [upload=59703]: Done uploading segments... Aug 11 02:42:32.193151548 (57h16m26.624231679s) [upload=59703]: Upload finished. err: <nil> Duration:* 1.834626721s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.....X....................] duration: 8.951040036s FAIL (2025-08-11 02:43:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 8.951065315s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.073872366s FAIL (2025-08-11 02:42:57 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.073895189s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.....X...................] duration: 7.624304012s FAIL (2025-08-11 02:42:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 7.624328188s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.124982539s FAIL (2025-08-11 02:42:39 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.125003298s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [....XX.......................................X.........X...X] duration: 1.370577974s FAIL (2025-08-11 02:42:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:42:31.914140022 (57h16m26.345220143s) [upload=59703]: Starting upload Aug 11 02:42:31.914204895 (57h16m26.345285026s) [upload=59703]: Done starting upload Aug 11 02:42:31.914293916 (57h16m26.345374047s) [upload=59703]: Uploading segments... Aug 11 02:42:31.914330556 (57h16m26.345410687s) [upload=59703]: Got next segment. Inline: true Aug 11 02:42:31.914337289 (57h16m26.34541747s) [upload=59703]: Waiting for error group managing segments... Aug 11 02:42:31.914397023 (57h16m26.345477204s) [upload=59703]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:42:31.914404247 (57h16m26.345484389s) [upload=59703]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:42:31.914408556 (57h16m26.345488697s) [upload=59703]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:42:32.193134135 (57h16m26.624214266s) [upload=59703]: Done uploading segments... Aug 11 02:42:32.193151548 (57h16m26.624231679s) [upload=59703]: Upload finished. err: <nil> Duration:* 1.370610567s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.....X..................] duration: 7.630073455s FAIL (2025-08-11 02:42:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 7.630103874s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [X....XX.......................................X.........X...] duration: 1.8745394s FAIL (2025-08-11 02:41:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:40:25.430699467 (57h14m19.861779598s) [upload=59664]: Starting upload Aug 11 02:40:25.430746678 (57h14m19.861826809s) [upload=59664]: Done starting upload Aug 11 02:40:25.430864443 (57h14m19.861944564s) [upload=59664]: Uploading segments... Aug 11 02:40:25.430904600 (57h14m19.861984731s) [upload=59664]: Got next segment. Inline: true Aug 11 02:40:25.430908818 (57h14m19.861988949s) [upload=59664]: Waiting for error group managing segments... Aug 11 02:40:25.430945438 (57h14m19.862025569s) [upload=59664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:40:25.430949596 (57h14m19.862029727s) [upload=59664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:40:25.430952040 (57h14m19.862032161s) [upload=59664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:40:25.723416257 (57h14m20.154496388s) [upload=59664]: Done uploading segments... Aug 11 02:40:25.723434543 (57h14m20.154514674s) [upload=59664]: Upload finished. err: <nil> Duration:* 1.874572433s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.268268276s FAIL (2025-08-11 02:41:58 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.268286328s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [XX....XX.......................................X.........X..] duration: 1.846010031s FAIL (2025-08-11 02:41:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:40:25.430699467 (57h14m19.861779598s) [upload=59664]: Starting upload Aug 11 02:40:25.430746678 (57h14m19.861826809s) [upload=59664]: Done starting upload Aug 11 02:40:25.430864443 (57h14m19.861944564s) [upload=59664]: Uploading segments... Aug 11 02:40:25.430904600 (57h14m19.861984731s) [upload=59664]: Got next segment. Inline: true Aug 11 02:40:25.430908818 (57h14m19.861988949s) [upload=59664]: Waiting for error group managing segments... Aug 11 02:40:25.430945438 (57h14m19.862025569s) [upload=59664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:40:25.430949596 (57h14m19.862029727s) [upload=59664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:40:25.430952040 (57h14m19.862032161s) [upload=59664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:40:25.723416257 (57h14m20.154496388s) [upload=59664]: Done uploading segments... Aug 11 02:40:25.723434543 (57h14m20.154514674s) [upload=59664]: Upload finished. err: <nil> Duration:* 1.846040248s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.....X.................] duration: 7.460961204s FAIL (2025-08-11 02:41:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 7.460993265s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [.XX....XX.......................................X.........X.] duration: 1.83557268s FAIL (2025-08-11 02:41:03 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:40:25.430699467 (57h14m19.861779598s) [upload=59664]: Starting upload Aug 11 02:40:25.430746678 (57h14m19.861826809s) [upload=59664]: Done starting upload Aug 11 02:40:25.430864443 (57h14m19.861944564s) [upload=59664]: Uploading segments... Aug 11 02:40:25.430904600 (57h14m19.861984731s) [upload=59664]: Got next segment. Inline: true Aug 11 02:40:25.430908818 (57h14m19.861988949s) [upload=59664]: Waiting for error group managing segments... Aug 11 02:40:25.430945438 (57h14m19.862025569s) [upload=59664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:40:25.430949596 (57h14m19.862029727s) [upload=59664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:40:25.430952040 (57h14m19.862032161s) [upload=59664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:40:25.723416257 (57h14m20.154496388s) [upload=59664]: Done uploading segments... Aug 11 02:40:25.723434543 (57h14m20.154514674s) [upload=59664]: Upload finished. err: <nil> Duration:* 1.835597107s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.092746069s FAIL (2025-08-11 02:41:02 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.092766217s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.....X................] duration: 7.929673779s FAIL (2025-08-11 02:40:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 7.929726069s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.18452668s FAIL (2025-08-11 02:40:38 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.184553321s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [..XX....XX.......................................X.........X] duration: 1.391285036s FAIL (2025-08-11 02:40:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:40:25.430699467 (57h14m19.861779598s) [upload=59664]: Starting upload Aug 11 02:40:25.430746678 (57h14m19.861826809s) [upload=59664]: Done starting upload Aug 11 02:40:25.430864443 (57h14m19.861944564s) [upload=59664]: Uploading segments... Aug 11 02:40:25.430904600 (57h14m19.861984731s) [upload=59664]: Got next segment. Inline: true Aug 11 02:40:25.430908818 (57h14m19.861988949s) [upload=59664]: Waiting for error group managing segments... Aug 11 02:40:25.430945438 (57h14m19.862025569s) [upload=59664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:40:25.430949596 (57h14m19.862029727s) [upload=59664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:40:25.430952040 (57h14m19.862032161s) [upload=59664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:40:25.723416257 (57h14m20.154496388s) [upload=59664]: Done uploading segments... Aug 11 02:40:25.723434543 (57h14m20.154514674s) [upload=59664]: Upload finished. err: <nil> Duration:* 1.391313962s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.152328344s FAIL (2025-08-11 02:39:46 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.152350296s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.....X...............] duration: 13.957156166s FAIL (2025-08-11 02:39:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 13.957178818s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...XX....XX.......................................X.........] duration: 1.80580953s FAIL (2025-08-11 02:39:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 02:32:55.861008535 (57h6m50.292088656s) [upload=59541]: Starting upload Aug 11 02:32:55.861066355 (57h6m50.292146486s) [upload=59541]: Done starting upload Aug 11 02:32:55.861135908 (57h6m50.292216029s) [upload=59541]: Uploading segments... Aug 11 02:32:55.861245778 (57h6m50.292325909s) [upload=59541]: Got next segment. Inline: true Aug 11 02:32:55.861251198 (57h6m50.292331329s) [upload=59541]: Waiting for error group managing segments... Aug 11 02:32:55.861292638 (57h6m50.292372759s) [upload=59541]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:32:55.861297307 (57h6m50.292377438s) [upload=59541]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:32:55.861299942 (57h6m50.292380073s) [upload=59541]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:32:56.074589156 (57h6m50.505669287s) [upload=59541]: Done uploading segments... Aug 11 02:32:56.074607421 (57h6m50.505687552s) [upload=59541]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.805848485s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.....X..............] duration: 9.59741211s FAIL (2025-08-11 02:39:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 9.59744253s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.....X.............] duration: 7.645917465s FAIL (2025-08-11 02:38:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 7.645945969s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)