Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X.....X.] duration: 7.407622583s FAIL (2025-08-11 02:29:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 7.407649995s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.191803814s FAIL (2025-08-11 02:28:54 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.191825264s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.................XX....XX...................................] duration: 1.979842287s FAIL (2025-08-11 02:28:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.979871022s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..................XX....XX..................................] duration: 1.84925243s FAIL (2025-08-11 02:28:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.849280331s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.159191019s FAIL (2025-08-11 02:28:27 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.159212819s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X.....X] duration: 9.122191924s FAIL (2025-08-11 02:28:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 58f01ece630c3404f89598722d78d1f1e49645a4db5a11b94b9864a901cc2fcc != before 8eaaf96f84cdab2584eae7128d0770b935d0d4996a16c0d26568f5f645f08109 ======LOGS====== Aug 11 02:27:55.488794605 (57h1m49.919874736s) [upload=59459]: Starting upload Aug 11 02:27:55.488847607 (57h1m49.919927728s) [upload=59459]: Done starting upload Aug 11 02:27:55.488921138 (57h1m49.920001269s) [upload=59459]: Uploading segments... Aug 11 02:27:55.489097234 (57h1m49.920177355s) [upload=59459]: Got next segment. Inline: false Aug 11 02:27:55.489128364 (57h1m49.920208495s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:27:55.489133284 (57h1m49.920213405s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:27:55.943654193 (57h1m50.374734315s) [seg_pos={0 0}, upload=59459]: Begin upload segment... Aug 11 02:27:55.943702186 (57h1m50.374782317s) [seg_pos={0 0}, upload=59459]: Done begin upload segment. Aug 11 02:27:55.943779233 (57h1m50.374859354s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.944715014 (57h1m50.375795145s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945416887 (57h1m50.376497018s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.945724946 (57h1m50.376805077s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946023007 (57h1m50.377103128s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.946307250 (57h1m50.377387371s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947059560 (57h1m50.378139681s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947292024 (57h1m50.378372155s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947510503 (57h1m50.378590634s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947746094 (57h1m50.378826225s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.947973869 (57h1m50.379054s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.950446981 (57h1m50.381527102s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.952835671 (57h1m50.383915803s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953220848 (57h1m50.384300979s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.953890610 (57h1m50.384970731s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.954391709 (57h1m50.38547184s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.955678471 (57h1m50.386758602s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:55.956814995 (57h1m50.387895126s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Uploading piece... Aug 11 02:27:57.125816745 (57h1m51.556896876s) [upload=59459]: Next returned nil segment Aug 11 02:27:57.125835300 (57h1m51.556915431s) [upload=59459]: Waiting for error group managing segments... Aug 11 02:27:57.425454656 (57h1m51.856534787s) [noise=true , address=64.49.13.169:4707 , piece_id=CKI5JTQEJ47JM4QTA3W7G2M5B6FUR4DYYCATM2LB5AKESHVIA7AA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.520393588 (57h1m52.951473719s) [noise=true , address=104.36.82.23:4455 , piece_id=H6E7JZ6VLJRN7KIU4BFQSL3GKX4OIQQO2K35ZLBFOWHYZ326T5TQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.604019675 (57h1m53.035099816s) [noise=true , address=104.36.82.4:4821 , piece_id=YNVVF6A47M67WU75E7BMDX22C43XQ6IS7TAATQ3YNOR35P3UFBKQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.701180380 (57h1m53.132260511s) [noise=true , address=104.36.82.51:4049 , piece_id=DV6PU24CYD52FEIXTBKZQ6F5RXKAB2L67XBH5CJRFNT7W5Q44EYQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.834316090 (57h1m53.265396221s) [noise=true , address=104.36.82.62:4425 , piece_id=SN7KQZXGHHVBV5TFI73SQJGFNFVKAAFKF3RKIG3EJHNPGUJPEVNA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.843352820 (57h1m53.274432951s) [noise=true , address=104.36.82.34:4545 , piece_id=CGXHQKZTP5DGES2O5MD2XSEYUORZDAI4BBBXCWUQNADXMRKB7IGQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.862488113 (57h1m53.293568244s) [noise=true , address=104.36.82.19:4433 , piece_id=4YNMAXROZLV7C3HYVCJVCDD6MIQHY7RH3PQPXSF3RGLD4A5GRSHQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.871883370 (57h1m53.302963521s) [noise=true , address=104.36.82.38:4716 , piece_id=VD42BEHBOKSOOQ7YF7B7LFM2DR2QKQ2D7ZZICI7RKU7YN4O6ZDZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.889107466 (57h1m53.320187587s) [noise=true , address=104.36.82.58:4280 , piece_id=O42DLN6H4CS35B3UXM2W6GSLBTSBI43IQIQEJA64AZHT6BYZSVZA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:58.891189069 (57h1m53.3222692s) [noise=true , address=104.36.82.39:4757 , piece_id=IO4R3DRG47GAM73MKRF3C6H22VIBQLKYZ5LCXWZX6IVCSJOXILBA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.338987425 (57h1m53.770067556s) [noise=true , address=104.36.82.37:4653 , piece_id=UFS7YC43F4YZMAF3M3NBPSHKOZQSL3YSQETRUSMLQBUNBBZY2PXA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.379269370 (57h1m53.810349501s) [noise=true , address=104.36.82.13:4213 , piece_id=YKBUCD5TEJTZRE6LOB7TJQOMMJZR7Z66EHQQOKHRH5JWBJTACPQA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.540777035 (57h1m53.971857167s) [noise=true , address=104.36.82.37:4673 , piece_id=HWHCOW7ZUDH56ASRLPZ2XJOSFQOJYZDIFF25ZDD7OCVSTCELYXUQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742853102 (57h1m54.173933223s) [noise=true , address=104.36.82.36:4641 , piece_id=CTGDF36TY7UVYJE5KT6A3JHNAYL2HSGULSQCK3SJTMFFU7MZIIPA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: <nil> Aug 11 02:27:59.742880344 (57h1m54.173960466s) [seg_pos={0 0}, upload=59459]: Segment reached optimal threshold of 14 pieces. Aug 11 02:27:59.743085947 (57h1m54.174166088s) [noise=true , address=104.36.82.47:4525 , piece_id=BSVGQDFKOH2A3NM74HYPKRMNICY7UPNTJHNASCXTRY6VYXYKXO7Q, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743175869 (57h1m54.174256s) [noise=true , address=104.36.82.2:4763 , piece_id=ZFMRANXPLEKCUTOMDRMRPZAZQVVAMGEOJ5ETYSBQZHYWEOBMK5LQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743195938 (57h1m54.174276069s) [noise=true , address=104.36.82.5:4858 , piece_id=DPRK5PKN52EVGQUU7YLRJNAYCGO32DTQ3J5QAC24GF5ZKCEU23GA, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36914->104.36.82.5:4858: use of closed network connection Aug 11 02:27:59.743262075 (57h1m54.174342206s) [noise=true , address=104.36.82.60:4366 , piece_id=CC22UE3ZDTEHVNRQLK6EGBBKH67ECFAUF7FJAPDDN6R3C7IJQASQ, seg_pos={0 0}, upload=59459]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NBUjBGU2MCjDx8iZKvzCvhL6GHCfdXfihC5H2uoBUmzhz9uoV): protocol: expected piece hash; context canceled; EOF Aug 11 02:27:59.743283515 (57h1m54.174363636s) [seg_pos={0 0}, upload=59459]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:27:59.743354622 (57h1m54.174434793s) [upload=59459]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:27:59.743361805 (57h1m54.174441986s) [upload=59459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:28:00.198203750 (57h1m54.629283881s) [upload=59459]: Done uploading segments... Aug 11 02:28:00.198227035 (57h1m54.629307176s) [upload=59459]: Upload finished. err: <nil> Duration: 9.122213545s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...................XX....XX.................................] duration: 1.838729764s FAIL (2025-08-11 02:28:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.838761614s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.265322578s FAIL (2025-08-11 02:27:42 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.265336794s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [....................XX....XX................................] duration: 1.843999728s FAIL (2025-08-11 02:27:35 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.844020928s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.875382698s FAIL (2025-08-11 02:27:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c31344cccbe2d388db5ada1dad4d65ba5a68d501fd333cef4f7c020ce6f6b6f9 != before 993e90b1076904292c2236cf2055806b1bf80cbd3253e15790115388106d7b29 ======LOGS====== Aug 11 02:23:03.073930355 (56h56m57.505010486s) [upload=59380]: Starting upload Aug 11 02:23:03.073977465 (56h56m57.505057586s) [upload=59380]: Done starting upload Aug 11 02:23:03.074054583 (56h56m57.505134724s) [upload=59380]: Uploading segments... Aug 11 02:23:03.074214639 (56h56m57.505294821s) [upload=59380]: Got next segment. Inline: false Aug 11 02:23:03.074243114 (56h56m57.505323245s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:23:03.074247552 (56h56m57.505327683s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:23:03.600716624 (56h56m58.031796755s) [seg_pos={0 0}, upload=59380]: Begin upload segment... Aug 11 02:23:03.600768894 (56h56m58.031849025s) [seg_pos={0 0}, upload=59380]: Done begin upload segment. Aug 11 02:23:03.600928699 (56h56m58.03200884s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.601241588 (56h56m58.032321719s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602029166 (56h56m58.033109297s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602177891 (56h56m58.033258022s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602747320 (56h56m58.033827452s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603296822 (56h56m58.034376953s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603508587 (56h56m58.034588718s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604223245 (56h56m58.035303376s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604632187 (56h56m58.035712318s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605060207 (56h56m58.036140338s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605211086 (56h56m58.036291217s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606019101 (56h56m58.037099232s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606299158 (56h56m58.037379289s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606683794 (56h56m58.037763925s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606950314 (56h56m58.038030445s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.607802244 (56h56m58.038882365s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608339733 (56h56m58.039419854s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608927198 (56h56m58.040007359s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:04.762231340 (56h56m59.193311471s) [upload=59380]: Next returned nil segment Aug 11 02:23:04.762246450 (56h56m59.193326581s) [upload=59380]: Waiting for error group managing segments... Aug 11 02:23:05.055194922 (56h56m59.486275053s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:05.577353233 (56h57m0.008433364s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.724387007 (56h57m1.155467128s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.855551108 (56h57m1.286631239s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.003929123 (56h57m1.435009254s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.045519218 (56h57m1.476599349s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.126162151 (56h57m1.557242282s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.206825124 (56h57m1.637905255s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.269056651 (56h57m1.700136782s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.274343158 (56h57m1.705423289s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.279082599 (56h57m1.71016273s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.296877074 (56h57m1.727957205s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.374111765 (56h57m1.805191896s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399623308 (56h57m1.830703439s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399707440 (56h57m1.830787571s) [seg_pos={0 0}, upload=59380]: Segment reached optimal threshold of 14 pieces. Aug 11 02:23:07.399841617 (56h57m1.830921748s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UEUsbCaVzugWQ3VWyyqpxRSzWxsaHqS8hzN8srq4mrxs1bvQJ): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.399915257 (56h57m1.830995388s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.400015900 (56h57m1.831096031s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp): protocol: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection Aug 11 02:23:07.400051869 (56h57m1.831132s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12v2vJnE92VrJXQrbxnAxANHCVASuXiBKMUZECMVmEWEjsZRtsQ): protocol: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection Aug 11 02:23:07.400073611 (56h57m1.831153742s) [seg_pos={0 0}, upload=59380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:23:07.400183791 (56h57m1.831263922s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:23:07.400190274 (56h57m1.831270405s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:23:07.703453456 (56h57m2.134533588s) [upload=59380]: Done uploading segments... Aug 11 02:23:07.703472263 (56h57m2.134552394s) [upload=59380]: Upload finished. err: <nil> Duration: 6.875407076s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.....................XX....XX...............................] duration: 1.852917831s FAIL (2025-08-11 02:27:01 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.852944784s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.504341974s FAIL (2025-08-11 02:26:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c31344cccbe2d388db5ada1dad4d65ba5a68d501fd333cef4f7c020ce6f6b6f9 != before 993e90b1076904292c2236cf2055806b1bf80cbd3253e15790115388106d7b29 ======LOGS====== Aug 11 02:23:03.073930355 (56h56m57.505010486s) [upload=59380]: Starting upload Aug 11 02:23:03.073977465 (56h56m57.505057586s) [upload=59380]: Done starting upload Aug 11 02:23:03.074054583 (56h56m57.505134724s) [upload=59380]: Uploading segments... Aug 11 02:23:03.074214639 (56h56m57.505294821s) [upload=59380]: Got next segment. Inline: false Aug 11 02:23:03.074243114 (56h56m57.505323245s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:23:03.074247552 (56h56m57.505327683s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:23:03.600716624 (56h56m58.031796755s) [seg_pos={0 0}, upload=59380]: Begin upload segment... Aug 11 02:23:03.600768894 (56h56m58.031849025s) [seg_pos={0 0}, upload=59380]: Done begin upload segment. Aug 11 02:23:03.600928699 (56h56m58.03200884s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.601241588 (56h56m58.032321719s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602029166 (56h56m58.033109297s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602177891 (56h56m58.033258022s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602747320 (56h56m58.033827452s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603296822 (56h56m58.034376953s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603508587 (56h56m58.034588718s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604223245 (56h56m58.035303376s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604632187 (56h56m58.035712318s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605060207 (56h56m58.036140338s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605211086 (56h56m58.036291217s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606019101 (56h56m58.037099232s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606299158 (56h56m58.037379289s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606683794 (56h56m58.037763925s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606950314 (56h56m58.038030445s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.607802244 (56h56m58.038882365s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608339733 (56h56m58.039419854s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608927198 (56h56m58.040007359s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:04.762231340 (56h56m59.193311471s) [upload=59380]: Next returned nil segment Aug 11 02:23:04.762246450 (56h56m59.193326581s) [upload=59380]: Waiting for error group managing segments... Aug 11 02:23:05.055194922 (56h56m59.486275053s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:05.577353233 (56h57m0.008433364s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.724387007 (56h57m1.155467128s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.855551108 (56h57m1.286631239s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.003929123 (56h57m1.435009254s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.045519218 (56h57m1.476599349s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.126162151 (56h57m1.557242282s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.206825124 (56h57m1.637905255s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.269056651 (56h57m1.700136782s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.274343158 (56h57m1.705423289s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.279082599 (56h57m1.71016273s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.296877074 (56h57m1.727957205s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.374111765 (56h57m1.805191896s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399623308 (56h57m1.830703439s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399707440 (56h57m1.830787571s) [seg_pos={0 0}, upload=59380]: Segment reached optimal threshold of 14 pieces. Aug 11 02:23:07.399841617 (56h57m1.830921748s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UEUsbCaVzugWQ3VWyyqpxRSzWxsaHqS8hzN8srq4mrxs1bvQJ): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.399915257 (56h57m1.830995388s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.400015900 (56h57m1.831096031s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp): protocol: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection Aug 11 02:23:07.400051869 (56h57m1.831132s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12v2vJnE92VrJXQrbxnAxANHCVASuXiBKMUZECMVmEWEjsZRtsQ): protocol: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection Aug 11 02:23:07.400073611 (56h57m1.831153742s) [seg_pos={0 0}, upload=59380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:23:07.400183791 (56h57m1.831263922s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:23:07.400190274 (56h57m1.831270405s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:23:07.703453456 (56h57m2.134533588s) [upload=59380]: Done uploading segments... Aug 11 02:23:07.703472263 (56h57m2.134552394s) [upload=59380]: Upload finished. err: <nil> Duration: 6.504368215s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.008753086s FAIL (2025-08-11 02:26:53 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.008771371s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [......................XX....XX..............................] duration: 1.8571472s FAIL (2025-08-11 02:26:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.857172639s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.......................XX....XX.............................] duration: 1.908756161s FAIL (2025-08-11 02:25:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.908781288s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 7.536988751s FAIL (2025-08-11 02:25:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c31344cccbe2d388db5ada1dad4d65ba5a68d501fd333cef4f7c020ce6f6b6f9 != before 993e90b1076904292c2236cf2055806b1bf80cbd3253e15790115388106d7b29 ======LOGS====== Aug 11 02:23:03.073930355 (56h56m57.505010486s) [upload=59380]: Starting upload Aug 11 02:23:03.073977465 (56h56m57.505057586s) [upload=59380]: Done starting upload Aug 11 02:23:03.074054583 (56h56m57.505134724s) [upload=59380]: Uploading segments... Aug 11 02:23:03.074214639 (56h56m57.505294821s) [upload=59380]: Got next segment. Inline: false Aug 11 02:23:03.074243114 (56h56m57.505323245s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:23:03.074247552 (56h56m57.505327683s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:23:03.600716624 (56h56m58.031796755s) [seg_pos={0 0}, upload=59380]: Begin upload segment... Aug 11 02:23:03.600768894 (56h56m58.031849025s) [seg_pos={0 0}, upload=59380]: Done begin upload segment. Aug 11 02:23:03.600928699 (56h56m58.03200884s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.601241588 (56h56m58.032321719s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602029166 (56h56m58.033109297s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602177891 (56h56m58.033258022s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602747320 (56h56m58.033827452s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603296822 (56h56m58.034376953s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603508587 (56h56m58.034588718s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604223245 (56h56m58.035303376s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604632187 (56h56m58.035712318s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605060207 (56h56m58.036140338s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605211086 (56h56m58.036291217s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606019101 (56h56m58.037099232s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606299158 (56h56m58.037379289s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606683794 (56h56m58.037763925s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606950314 (56h56m58.038030445s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.607802244 (56h56m58.038882365s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608339733 (56h56m58.039419854s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608927198 (56h56m58.040007359s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:04.762231340 (56h56m59.193311471s) [upload=59380]: Next returned nil segment Aug 11 02:23:04.762246450 (56h56m59.193326581s) [upload=59380]: Waiting for error group managing segments... Aug 11 02:23:05.055194922 (56h56m59.486275053s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:05.577353233 (56h57m0.008433364s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.724387007 (56h57m1.155467128s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.855551108 (56h57m1.286631239s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.003929123 (56h57m1.435009254s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.045519218 (56h57m1.476599349s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.126162151 (56h57m1.557242282s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.206825124 (56h57m1.637905255s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.269056651 (56h57m1.700136782s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.274343158 (56h57m1.705423289s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.279082599 (56h57m1.71016273s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.296877074 (56h57m1.727957205s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.374111765 (56h57m1.805191896s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399623308 (56h57m1.830703439s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399707440 (56h57m1.830787571s) [seg_pos={0 0}, upload=59380]: Segment reached optimal threshold of 14 pieces. Aug 11 02:23:07.399841617 (56h57m1.830921748s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UEUsbCaVzugWQ3VWyyqpxRSzWxsaHqS8hzN8srq4mrxs1bvQJ): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.399915257 (56h57m1.830995388s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.400015900 (56h57m1.831096031s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp): protocol: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection Aug 11 02:23:07.400051869 (56h57m1.831132s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12v2vJnE92VrJXQrbxnAxANHCVASuXiBKMUZECMVmEWEjsZRtsQ): protocol: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection Aug 11 02:23:07.400073611 (56h57m1.831153742s) [seg_pos={0 0}, upload=59380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:23:07.400183791 (56h57m1.831263922s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:23:07.400190274 (56h57m1.831270405s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:23:07.703453456 (56h57m2.134533588s) [upload=59380]: Done uploading segments... Aug 11 02:23:07.703472263 (56h57m2.134552394s) [upload=59380]: Upload finished. err: <nil> Duration: 7.537010383s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.622920755s FAIL (2025-08-11 02:25:54 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.6229656s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [........................XX....XX............................] duration: 1.820600874s FAIL (2025-08-11 02:25:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.820627324s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.261477292s FAIL (2025-08-11 02:25:03 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.261499834s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 12.51662088s FAIL (2025-08-11 02:25:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c31344cccbe2d388db5ada1dad4d65ba5a68d501fd333cef4f7c020ce6f6b6f9 != before 993e90b1076904292c2236cf2055806b1bf80cbd3253e15790115388106d7b29 ======LOGS====== Aug 11 02:23:03.073930355 (56h56m57.505010486s) [upload=59380]: Starting upload Aug 11 02:23:03.073977465 (56h56m57.505057586s) [upload=59380]: Done starting upload Aug 11 02:23:03.074054583 (56h56m57.505134724s) [upload=59380]: Uploading segments... Aug 11 02:23:03.074214639 (56h56m57.505294821s) [upload=59380]: Got next segment. Inline: false Aug 11 02:23:03.074243114 (56h56m57.505323245s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:23:03.074247552 (56h56m57.505327683s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:23:03.600716624 (56h56m58.031796755s) [seg_pos={0 0}, upload=59380]: Begin upload segment... Aug 11 02:23:03.600768894 (56h56m58.031849025s) [seg_pos={0 0}, upload=59380]: Done begin upload segment. Aug 11 02:23:03.600928699 (56h56m58.03200884s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.601241588 (56h56m58.032321719s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602029166 (56h56m58.033109297s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602177891 (56h56m58.033258022s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602747320 (56h56m58.033827452s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603296822 (56h56m58.034376953s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603508587 (56h56m58.034588718s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604223245 (56h56m58.035303376s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604632187 (56h56m58.035712318s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605060207 (56h56m58.036140338s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605211086 (56h56m58.036291217s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606019101 (56h56m58.037099232s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606299158 (56h56m58.037379289s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606683794 (56h56m58.037763925s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606950314 (56h56m58.038030445s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.607802244 (56h56m58.038882365s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608339733 (56h56m58.039419854s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608927198 (56h56m58.040007359s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:04.762231340 (56h56m59.193311471s) [upload=59380]: Next returned nil segment Aug 11 02:23:04.762246450 (56h56m59.193326581s) [upload=59380]: Waiting for error group managing segments... Aug 11 02:23:05.055194922 (56h56m59.486275053s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:05.577353233 (56h57m0.008433364s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.724387007 (56h57m1.155467128s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.855551108 (56h57m1.286631239s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.003929123 (56h57m1.435009254s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.045519218 (56h57m1.476599349s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.126162151 (56h57m1.557242282s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.206825124 (56h57m1.637905255s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.269056651 (56h57m1.700136782s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.274343158 (56h57m1.705423289s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.279082599 (56h57m1.71016273s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.296877074 (56h57m1.727957205s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.374111765 (56h57m1.805191896s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399623308 (56h57m1.830703439s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399707440 (56h57m1.830787571s) [seg_pos={0 0}, upload=59380]: Segment reached optimal threshold of 14 pieces. Aug 11 02:23:07.399841617 (56h57m1.830921748s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UEUsbCaVzugWQ3VWyyqpxRSzWxsaHqS8hzN8srq4mrxs1bvQJ): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.399915257 (56h57m1.830995388s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.400015900 (56h57m1.831096031s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp): protocol: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection Aug 11 02:23:07.400051869 (56h57m1.831132s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12v2vJnE92VrJXQrbxnAxANHCVASuXiBKMUZECMVmEWEjsZRtsQ): protocol: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection Aug 11 02:23:07.400073611 (56h57m1.831153742s) [seg_pos={0 0}, upload=59380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:23:07.400183791 (56h57m1.831263922s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:23:07.400190274 (56h57m1.831270405s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:23:07.703453456 (56h57m2.134533588s) [upload=59380]: Done uploading segments... Aug 11 02:23:07.703472263 (56h57m2.134552394s) [upload=59380]: Upload finished. err: <nil> Duration: 12.516651969s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.........................XX....XX...........................] duration: 1.997795955s FAIL (2025-08-11 02:24:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.997818899s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.139823162s FAIL (2025-08-11 02:24:45 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.139844624s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........................XX....XX..........................] duration: 1.908071883s FAIL (2025-08-11 02:24:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.908090319s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...........................XX....XX.........................] duration: 1.851328494s FAIL (2025-08-11 02:24:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.851355184s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 12.273137281s FAIL (2025-08-11 02:24:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c31344cccbe2d388db5ada1dad4d65ba5a68d501fd333cef4f7c020ce6f6b6f9 != before 993e90b1076904292c2236cf2055806b1bf80cbd3253e15790115388106d7b29 ======LOGS====== Aug 11 02:23:03.073930355 (56h56m57.505010486s) [upload=59380]: Starting upload Aug 11 02:23:03.073977465 (56h56m57.505057586s) [upload=59380]: Done starting upload Aug 11 02:23:03.074054583 (56h56m57.505134724s) [upload=59380]: Uploading segments... Aug 11 02:23:03.074214639 (56h56m57.505294821s) [upload=59380]: Got next segment. Inline: false Aug 11 02:23:03.074243114 (56h56m57.505323245s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:23:03.074247552 (56h56m57.505327683s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:23:03.600716624 (56h56m58.031796755s) [seg_pos={0 0}, upload=59380]: Begin upload segment... Aug 11 02:23:03.600768894 (56h56m58.031849025s) [seg_pos={0 0}, upload=59380]: Done begin upload segment. Aug 11 02:23:03.600928699 (56h56m58.03200884s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.601241588 (56h56m58.032321719s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602029166 (56h56m58.033109297s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602177891 (56h56m58.033258022s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602747320 (56h56m58.033827452s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603296822 (56h56m58.034376953s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603508587 (56h56m58.034588718s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604223245 (56h56m58.035303376s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604632187 (56h56m58.035712318s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605060207 (56h56m58.036140338s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605211086 (56h56m58.036291217s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606019101 (56h56m58.037099232s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606299158 (56h56m58.037379289s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606683794 (56h56m58.037763925s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606950314 (56h56m58.038030445s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.607802244 (56h56m58.038882365s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608339733 (56h56m58.039419854s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608927198 (56h56m58.040007359s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:04.762231340 (56h56m59.193311471s) [upload=59380]: Next returned nil segment Aug 11 02:23:04.762246450 (56h56m59.193326581s) [upload=59380]: Waiting for error group managing segments... Aug 11 02:23:05.055194922 (56h56m59.486275053s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:05.577353233 (56h57m0.008433364s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.724387007 (56h57m1.155467128s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.855551108 (56h57m1.286631239s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.003929123 (56h57m1.435009254s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.045519218 (56h57m1.476599349s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.126162151 (56h57m1.557242282s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.206825124 (56h57m1.637905255s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.269056651 (56h57m1.700136782s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.274343158 (56h57m1.705423289s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.279082599 (56h57m1.71016273s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.296877074 (56h57m1.727957205s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.374111765 (56h57m1.805191896s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399623308 (56h57m1.830703439s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399707440 (56h57m1.830787571s) [seg_pos={0 0}, upload=59380]: Segment reached optimal threshold of 14 pieces. Aug 11 02:23:07.399841617 (56h57m1.830921748s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UEUsbCaVzugWQ3VWyyqpxRSzWxsaHqS8hzN8srq4mrxs1bvQJ): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.399915257 (56h57m1.830995388s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.400015900 (56h57m1.831096031s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp): protocol: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection Aug 11 02:23:07.400051869 (56h57m1.831132s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12v2vJnE92VrJXQrbxnAxANHCVASuXiBKMUZECMVmEWEjsZRtsQ): protocol: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection Aug 11 02:23:07.400073611 (56h57m1.831153742s) [seg_pos={0 0}, upload=59380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:23:07.400183791 (56h57m1.831263922s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:23:07.400190274 (56h57m1.831270405s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:23:07.703453456 (56h57m2.134533588s) [upload=59380]: Done uploading segments... Aug 11 02:23:07.703472263 (56h57m2.134552394s) [upload=59380]: Upload finished. err: <nil> Duration: 12.273170665s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 7.092129248s FAIL (2025-08-11 02:23:56 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.092152452s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 7.808467518s FAIL (2025-08-11 02:23:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c31344cccbe2d388db5ada1dad4d65ba5a68d501fd333cef4f7c020ce6f6b6f9 != before 993e90b1076904292c2236cf2055806b1bf80cbd3253e15790115388106d7b29 ======LOGS====== Aug 11 02:23:03.073930355 (56h56m57.505010486s) [upload=59380]: Starting upload Aug 11 02:23:03.073977465 (56h56m57.505057586s) [upload=59380]: Done starting upload Aug 11 02:23:03.074054583 (56h56m57.505134724s) [upload=59380]: Uploading segments... Aug 11 02:23:03.074214639 (56h56m57.505294821s) [upload=59380]: Got next segment. Inline: false Aug 11 02:23:03.074243114 (56h56m57.505323245s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:23:03.074247552 (56h56m57.505327683s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 02:23:03.600716624 (56h56m58.031796755s) [seg_pos={0 0}, upload=59380]: Begin upload segment... Aug 11 02:23:03.600768894 (56h56m58.031849025s) [seg_pos={0 0}, upload=59380]: Done begin upload segment. Aug 11 02:23:03.600928699 (56h56m58.03200884s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.601241588 (56h56m58.032321719s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602029166 (56h56m58.033109297s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602177891 (56h56m58.033258022s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.602747320 (56h56m58.033827452s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603296822 (56h56m58.034376953s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.603508587 (56h56m58.034588718s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604223245 (56h56m58.035303376s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.604632187 (56h56m58.035712318s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605060207 (56h56m58.036140338s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.605211086 (56h56m58.036291217s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606019101 (56h56m58.037099232s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606299158 (56h56m58.037379289s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606683794 (56h56m58.037763925s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.606950314 (56h56m58.038030445s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.607802244 (56h56m58.038882365s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608339733 (56h56m58.039419854s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:03.608927198 (56h56m58.040007359s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Uploading piece... Aug 11 02:23:04.762231340 (56h56m59.193311471s) [upload=59380]: Next returned nil segment Aug 11 02:23:04.762246450 (56h56m59.193326581s) [upload=59380]: Waiting for error group managing segments... Aug 11 02:23:05.055194922 (56h56m59.486275053s) [noise=true , address=79.127.205.245:4222 , piece_id=VUZSKHQIRLU2HI5SY4GSL7WHXCFSD3V6GHGY24BRU3MLC3LNCRVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:05.577353233 (56h57m0.008433364s) [noise=true , address=79.127.201.215:4353 , piece_id=JBAXFEW3C22FLA4TGO7QBTWUYNRKXVKOX2O2JASNFCPC33YB6O7Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.724387007 (56h57m1.155467128s) [noise=true , address=104.36.82.61:4393 , piece_id=FCCHICDBC5FF4IKQPKNRVMRAMY7OP6B6APTJQWKYGHXMEMVC4G5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:06.855551108 (56h57m1.286631239s) [noise=true , address=104.36.82.37:4673 , piece_id=JG7VQHGQ3DNR3HA2XPAQ2KL6HW5ILPWXANDNIHYLGJY7TPEBWLEA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.003929123 (56h57m1.435009254s) [noise=true , address=104.36.82.37:4656 , piece_id=PTYX4OE4BFL2GQU5PBIAV2PH333IPZRPSWEHWVGHS7LJNPSOGZ5Q, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.045519218 (56h57m1.476599349s) [noise=true , address=104.36.82.38:4696 , piece_id=A2ENO5TBPPGPXI2FPE3GACRNAW2TEQQJ4MLYAZCTM7WYTU3SZKPA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.126162151 (56h57m1.557242282s) [noise=true , address=104.36.82.7:4939 , piece_id=DAR7AWAKFEILMXYHRWFEN6UXR3P7X5ZLIKR6QSMTWP6PUXJUBTNA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.206825124 (56h57m1.637905255s) [noise=true , address=104.36.82.35:4613 , piece_id=ZUIAUIDUQRCEAREV57MOCUVEIW3SIJVVETCPAJZTYQJHVXNXGKEQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.269056651 (56h57m1.700136782s) [noise=true , address=104.36.82.47:4552 , piece_id=73BJ4JFDYAN5UR3BBXFJCZRBN57TC7WUHCGIDTWSJ4XFFSGAJNSA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.274343158 (56h57m1.705423289s) [noise=true , address=104.36.82.60:4343 , piece_id=LM5M3Q3FYIBALX6ENUGIMDSGEE77XILJ4Z2AGI2AEMROPTA4DRZA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.279082599 (56h57m1.71016273s) [noise=true , address=104.36.82.5:4853 , piece_id=X5DOB3R6TFHT3X3NHOBGPRWHDE6HSJE7N36FH5ZK2PZ7SVDUAGNQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.296877074 (56h57m1.727957205s) [noise=true , address=104.36.82.62:4428 , piece_id=2VXQBQPAUGJ7EDSJTXJNZWDRMRNDANW6S5EMAH5NCGB2ZBKOMWVA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.374111765 (56h57m1.805191896s) [noise=true , address=104.36.82.17:4341 , piece_id=U2CJIB7CIEF6GUBDYOJFDOR7X6SCSJVPDITDRNBWELKNQVKLHM3A, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399623308 (56h57m1.830703439s) [noise=true , address=104.36.82.23:4472 , piece_id=6UC3SDFYK3RWZG5G7KRG6C5ANZAQ5FMYQWGAWXGSC2RJT23OBILA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: <nil> Aug 11 02:23:07.399707440 (56h57m1.830787571s) [seg_pos={0 0}, upload=59380]: Segment reached optimal threshold of 14 pieces. Aug 11 02:23:07.399841617 (56h57m1.830921748s) [noise=true , address=104.36.82.59:4332 , piece_id=XGGPSRY5D4FIVYWVOF5U47IXXRDL2VVQNKBFSG3PE4PYXS6CGKKA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UEUsbCaVzugWQ3VWyyqpxRSzWxsaHqS8hzN8srq4mrxs1bvQJ): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.399915257 (56h57m1.830995388s) [noise=true , address=104.36.82.20:4456 , piece_id=OWQD56PQGTMMA74NEZV7ILKM3WLLAAJLO7NCQVK6U5OJKWZLBDHA, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 11 02:23:07.400015900 (56h57m1.831096031s) [noise=true , address=104.36.82.36:4630 , piece_id=PXKQLLXJUZQIAON6KUAGGGUE42V7VZZOFINF4X5O6FQQTUIFS2QQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp): protocol: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50444->104.36.82.36:4630: use of closed network connection Aug 11 02:23:07.400051869 (56h57m1.831132s) [noise=true , address=104.36.82.58:4286 , piece_id=HXIBVTBZ6CLH3E2KLOWRVISQ2OU46NAMM3XQX3KG2NXLIL7DM4DQ, seg_pos={0 0}, upload=59380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12v2vJnE92VrJXQrbxnAxANHCVASuXiBKMUZECMVmEWEjsZRtsQ): protocol: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40110->104.36.82.58:4286: use of closed network connection Aug 11 02:23:07.400073611 (56h57m1.831153742s) [seg_pos={0 0}, upload=59380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 02:23:07.400183791 (56h57m1.831263922s) [upload=59380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 02:23:07.400190274 (56h57m1.831270405s) [upload=59380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:23:07.703453456 (56h57m2.134533588s) [upload=59380]: Done uploading segments... Aug 11 02:23:07.703472263 (56h57m2.134552394s) [upload=59380]: Upload finished. err: <nil> Duration: 7.808493819s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [............................XX....XX........................] duration: 1.971967886s FAIL (2025-08-11 02:23:09 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.971992472s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.181965181s FAIL (2025-08-11 02:22:56 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.182021338s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............................XX....XX.......................] duration: 1.8655952s FAIL (2025-08-11 02:22:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.865628193s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.002223364s FAIL (2025-08-11 02:21:52 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.002243221s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.109862735s FAIL (2025-08-11 02:21:15 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.109879818s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..............................XX....XX......................] duration: 1.867396418s FAIL (2025-08-11 02:20:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.867425823s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.989064304s FAIL (2025-08-11 02:20:19 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.989085576s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...............................XX....XX.....................] duration: 1.850892834s FAIL (2025-08-11 02:19:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.850920869s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.226805877s FAIL (2025-08-11 02:19:37 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.226821588s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [................................XX....XX....................] duration: 1.865976059s FAIL (2025-08-11 02:19:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.865997921s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.238531622s FAIL (2025-08-11 02:18:25 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.238555296s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.................................XX....XX...................] duration: 1.889297399s FAIL (2025-08-11 02:18:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.889386739s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.2471579s FAIL (2025-08-11 02:17:34 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.247179581s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..................................XX....XX..................] duration: 1.843999409s FAIL (2025-08-11 02:17:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.844027142s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.328893541s FAIL (2025-08-11 02:17:02 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.328915273s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 7.859351074s FAIL (2025-08-11 02:16:29 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.859372155s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...................................XX....XX.................] duration: 1.862677696s FAIL (2025-08-11 02:16:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.862696823s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.303863361s FAIL (2025-08-11 02:16:03 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.303908567s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [....................................XX....XX................] duration: 1.812196347s FAIL (2025-08-11 02:15:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 1.812222567s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.341320609s FAIL (2025-08-11 02:14:54 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.341632806s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.531477969s FAIL (2025-08-11 02:14:26 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.531494199s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.....................................XX....XX...............] duration: 2.037976635s FAIL (2025-08-11 02:14:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 02:03:51.208581050 (56h37m45.639661181s) [upload=59042]: Starting upload Aug 11 02:03:51.208624092 (56h37m45.639704223s) [upload=59042]: Done starting upload Aug 11 02:03:51.208673306 (56h37m45.639753437s) [upload=59042]: Uploading segments... Aug 11 02:03:51.208744402 (56h37m45.639824534s) [upload=59042]: Got next segment. Inline: true Aug 11 02:03:51.208748941 (56h37m45.639829062s) [upload=59042]: Waiting for error group managing segments... Aug 11 02:03:51.208787935 (56h37m45.639868066s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 02:03:51.208792003 (56h37m45.639872134s) [upload=59042]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 02:03:51.208794609 (56h37m45.63987473s) [upload=59042]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 02:03:51.478802669 (56h37m45.9098828s) [upload=59042]: Done uploading segments... Aug 11 02:03:51.478819391 (56h37m45.909899512s) [upload=59042]: Upload finished. err: <nil> Duration:* 2.037998366s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.999475742s FAIL (2025-08-11 02:13:55 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.999489589s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)