Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X..X.......X...........X...............X..............] duration: 5.929913372s FAIL (2025-08-13 03:23:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 5.929931617s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.90442367s FAIL (2025-08-13 03:23:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.904448317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X..........................................................X] duration: 855.422065ms FAIL (2025-08-13 03:23:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 855.438104ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................X.....XX.....] duration: 7.984546776s FAIL (2025-08-13 03:23:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.984561643s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X..X.......X...........X...............X.............] duration: 5.805187924s FAIL (2025-08-13 03:23:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 5.80521807s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [........X..X.......X...........X...............X............] duration: 6.467777051s FAIL (2025-08-13 03:23:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 6.467805595s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.863458887s FAIL (2025-08-13 03:22:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.863497031s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................................X.....XX....] duration: 7.964619462s FAIL (2025-08-13 03:22:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.964683154s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.866185827s FAIL (2025-08-13 03:22:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.86621329s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.925176766s FAIL (2025-08-13 03:22:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.925197987s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........X..X.......X...........X...............X...........] duration: 5.813012301s FAIL (2025-08-13 03:22:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 5.813056817s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.87557461s FAIL (2025-08-13 03:21:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.875599818s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X..X.......X...........X...............X..........] duration: 5.799760941s FAIL (2025-08-13 03:21:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 5.799780408s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................X.....XX...] duration: 7.260965953s FAIL (2025-08-13 03:21:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.260981903s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.842374678s FAIL (2025-08-13 03:21:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.842401619s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.845212028s FAIL (2025-08-13 03:21:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.845247627s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.879983158s FAIL (2025-08-13 03:21:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.880010509s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X..X.......X...........X...............X.........] duration: 5.615371372s FAIL (2025-08-13 03:21:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 5.615406259s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................X.....XX..] duration: 7.421103455s FAIL (2025-08-13 03:20:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.421124625s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.097697241s FAIL (2025-08-13 03:20:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.097724014s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................X.....XX.] duration: 14.235429552s FAIL (2025-08-13 03:20:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.235445493s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [............X..X.......X...........X...............X........] duration: 6.051299124s FAIL (2025-08-13 03:20:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 6.051325243s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.86000393s FAIL (2025-08-13 03:20:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.860034028s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................X.....XX] duration: 30.0009064s FAIL (2025-08-13 03:20:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000941116s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............X..X.......X...........X...............X.......] duration: 6.303852604s FAIL (2025-08-13 03:19:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 6.303888574s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.079127845s FAIL (2025-08-13 03:19:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.079152001s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............X..X.......X...........X...............X......] duration: 6.52667874s FAIL (2025-08-13 03:18:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 6.526708196s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.915856394s FAIL (2025-08-13 03:18:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.91588015s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.829020306s FAIL (2025-08-13 03:18:51 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.829040665s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X.....X] duration: 30.001531105s FAIL (2025-08-13 03:18:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001553077s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............X..X.......X...........X...............X.....] duration: 6.180255758s FAIL (2025-08-13 03:18:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 6.180289682s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 8.75084759s FAIL (2025-08-13 03:17:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.750859011s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.844248113s FAIL (2025-08-13 03:17:51 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.844276055s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.895256017s FAIL (2025-08-13 03:17:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.895344978s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 26.679357656s FAIL (2025-08-13 03:17:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 26.679378736s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X.......X...................................................] duration: 1.865411071s FAIL (2025-08-13 03:17:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.865441861s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.88447502s FAIL (2025-08-13 03:17:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.884504146s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [................X..X.......X...........X...............X....] duration: 8.059755819s FAIL (2025-08-13 03:17:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 8.059781428s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.081841036s FAIL (2025-08-13 03:17:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.081870903s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................X..X.......X...........X...............X...] duration: 6.187091687s FAIL (2025-08-13 03:16:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 6.187118088s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X.......X..................................................] duration: 1.962035718s FAIL (2025-08-13 03:16:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.962064203s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X.......X.................................................] duration: 1.821475573s FAIL (2025-08-13 03:16:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.821510218s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 9.813017968s FAIL (2025-08-13 03:16:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.813039048s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................X..X.......X...........X...............X..] duration: 7.358256223s FAIL (2025-08-13 03:16:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 7.358285179s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.877633771s FAIL (2025-08-13 03:16:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.877673407s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X.......X................................................] duration: 1.838315461s FAIL (2025-08-13 03:15:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.838349164s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 8.295176923s FAIL (2025-08-13 03:15:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.295200468s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 7.897824052s FAIL (2025-08-13 03:15:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.897842917s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................X..X.......X...........X...............X.] duration: 6.137007951s FAIL (2025-08-13 03:14:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 6.137031225s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X.......X...............................................] duration: 1.939873345s FAIL (2025-08-13 03:14:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.939906618s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)