Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.876124493s FAIL (2025-08-13 03:14:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.876156474s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000443558s FAIL (2025-08-13 03:14:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000459219s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....................X..X.......X...........X...............X] duration: 7.715576715s FAIL (2025-08-13 03:14:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 07a64daf5ce9db32d41aea6196fcc70ddc5670c6fb7b746f966c0a14f0f90cad != before e0a47ea874bf973a2cf983e7ff42ad385b8577ce8b4f3c6f68efa1aa67839818 ======LOGS====== Aug 13 03:14:27.200063120 (105h48m21.631143251s) [upload=110132]: Starting upload Aug 13 03:14:27.200107645 (105h48m21.631187776s) [upload=110132]: Done starting upload Aug 13 03:14:27.200237103 (105h48m21.631317234s) [upload=110132]: Uploading segments... Aug 13 03:14:27.200364997 (105h48m21.631445128s) [upload=110132]: Got next segment. Inline: false Aug 13 03:14:27.200415604 (105h48m21.631495785s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:14:27.200424100 (105h48m21.631504271s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:14:27.660811382 (105h48m22.091891513s) [seg_pos={0 0}, upload=110132]: Begin upload segment... Aug 13 03:14:27.660865877 (105h48m22.091946008s) [seg_pos={0 0}, upload=110132]: Done begin upload segment. Aug 13 03:14:27.660886587 (105h48m22.091966718s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.660900844 (105h48m22.091981055s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661384269 (105h48m22.0924644s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661598819 (105h48m22.09267894s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661669895 (105h48m22.092750026s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.661996209 (105h48m22.09307634s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662169181 (105h48m22.093249342s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662275895 (105h48m22.093356016s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662522045 (105h48m22.093602166s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662779709 (105h48m22.09385984s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.662985592 (105h48m22.094065723s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663176938 (105h48m22.094257069s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663367453 (105h48m22.094447584s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663568779 (105h48m22.09464891s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663767839 (105h48m22.09484797s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.663960167 (105h48m22.095040288s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664144580 (105h48m22.095224711s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:27.664326608 (105h48m22.095406739s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Uploading piece... Aug 13 03:14:29.202132828 (105h48m23.633212949s) [upload=110132]: Next returned nil segment Aug 13 03:14:29.202151514 (105h48m23.633231645s) [upload=110132]: Waiting for error group managing segments... Aug 13 03:14:29.624905241 (105h48m24.055985372s) [noise=true , address=77.90.31.14:4679 , piece_id=CFPI32OHVSZTT2YEOSL5ZXAHA37PT57VMTTTCTTIRQVMXAJM6D5A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.625120442 (105h48m24.056200574s) [noise=true , address=104.36.82.59:4312 , piece_id=FZB6SMU32MTZTBAHUTHLWVZGQ53C67FAAIIH6VE2MTLS6PGFXXZQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.667080856 (105h48m24.098160987s) [noise=true , address=104.36.82.9:4024 , piece_id=Y5CY7VLCSC6BNGIH6NF6446BOU4LNIX5GCCY6DSWCPKXPCK4QDQQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.669032951 (105h48m24.100113082s) [noise=true , address=104.36.82.14:4250 , piece_id=CQY4HBMBDYVUKR4637GSB33VW6G4OEP53VH27DMSBPS54CATSBWA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.699132833 (105h48m24.130212964s) [noise=true , address=104.36.82.11:4138 , piece_id=7L7M5DMAFNHOHVMUAQPFOKLU4TKIBL3TMS6D2F6X5T23BKDKGS7Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.717982878 (105h48m24.149063009s) [noise=true , address=104.36.82.36:4637 , piece_id=QXP45IJF7253J7A5MY3C4DAA4VFFT5XULJ3Y4QTXTDCTVJP4IYGA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.724446195 (105h48m24.155526316s) [noise=true , address=104.36.82.37:4678 , piece_id=6JTI7SDS4DTDJFAF4FZCCCOON7KOPOXXT5MZ2MDIPSQXZZKF6TKQ, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.731783564 (105h48m24.162863695s) [noise=true , address=104.36.82.58:4284 , piece_id=K5J626LGT3TOFW6Q4G22RJJYQGXPYS2B74RB34HVPD6HTOQA2QCA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.780951378 (105h48m24.212031509s) [noise=true , address=104.36.82.46:4484 , piece_id=YRE57GHVKPCOHVRDGSL4FGGX4NPXUJF4L7ELHZGHJ4EC4QDMEU3A, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:29.794319577 (105h48m24.225399708s) [noise=true , address=104.36.82.47:4544 , piece_id=TIKP5LMX65MWSBUUPMX36JYRVFC3JVCA5KLRS7CIZMF3SJVS73OA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.252036350 (105h48m24.683116481s) [noise=true , address=79.127.219.38:4035 , piece_id=MXSRDYRUPY6DSE6N66WNQZPCC6KVBHISPOIZBGFAQSD3CUGSFWOA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:30.523524634 (105h48m24.954604765s) [noise=true , address=104.36.82.23:4472 , piece_id=IRJTNTCAIGVWU4HIE3L3ZCZH6VM6FHFU3SEZDXFYEV4R2BAP45RA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.207986103 (105h48m25.639066235s) [noise=true , address=77.90.31.10:4528 , piece_id=XKSFJ2I7HYD72TOTWFHDRRLTWE6SFSMKTDVKYIYFWI6VYQF4EW5Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757453767 (105h48m26.188533898s) [noise=true , address=104.36.82.16:4299 , piece_id=CTNAIZNCPYLZLJZSH7OVSKBIJ2Q3J2QC75ONHCNPKVGRQEQWLL2Q, seg_pos={0 0}, upload=110132]: Done uploading piece. err: <nil> Aug 13 03:14:31.757477813 (105h48m26.188557934s) [seg_pos={0 0}, upload=110132]: Segment reached optimal threshold of 14 pieces. Aug 13 03:14:31.757561022 (105h48m26.188641153s) [noise=true , address=77.90.31.10:4530 , piece_id=JILOKHNOQCGIDXTL2NQF4HCNYE6RJXNH4OICDIVSHADVBIUVA4ZA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757627769 (105h48m26.1887079s) [noise=true , address=104.36.82.18:4377 , piece_id=Y7NIH46TSTZBIMNAK6WLRFX6WYAZR2BPQLVKHJ4OK44TTD3JP2AA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JHy4o8dxFNeeiitLPavxspcf4rL9KfckcRAuC121v8YZG6qmT): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757682684 (105h48m26.188762815s) [noise=true , address=104.36.82.17:4329 , piece_id=KPGYZK5QIGRNMUHQLFRGRNIPI5EONZBHCGARRSDZCK2UFJLNN4TA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Aug 13 03:14:31.757703594 (105h48m26.188783725s) [noise=true , address=104.36.82.60:4352 , piece_id=UCDAKY7PVPM2LXMCEZYWZNQJV6DJRZIBB2VMJP2IB6N4ZEU6Q7IA, seg_pos={0 0}, upload=110132]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47910->104.36.82.60:4352: use of closed network connection Aug 13 03:14:31.757749713 (105h48m26.188829844s) [seg_pos={0 0}, upload=110132]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:14:31.757833714 (105h48m26.188913845s) [upload=110132]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:14:31.757837731 (105h48m26.188917862s) [upload=110132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:14:32.059045060 (105h48m26.490125191s) [upload=110132]: Done uploading segments... Aug 13 03:14:32.059061040 (105h48m26.490141171s) [upload=110132]: Upload finished. err: <nil> Duration: 7.715609226s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X.......X..............................................] duration: 1.943568346s FAIL (2025-08-13 03:14:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.943593254s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.861950342s FAIL (2025-08-13 03:14:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.861979878s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................X..X.......X...........X...............] duration: 5.625329074s FAIL (2025-08-13 03:13:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.625355696s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X.......X.............................................] duration: 1.865894749s FAIL (2025-08-13 03:13:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.865919426s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.970664648s FAIL (2025-08-13 03:13:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.970687362s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................X..X.......X...........X..............] duration: 5.908952438s FAIL (2025-08-13 03:12:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.908987104s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.102195131s FAIL (2025-08-13 03:12:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.102224167s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X.......X............................................] duration: 1.868750153s FAIL (2025-08-13 03:12:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.868777535s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.829711483s FAIL (2025-08-13 03:12:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.829757209s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................X..X.......X...........X.............] duration: 6.008734947s FAIL (2025-08-13 03:12:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 6.008763711s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X.......X...........................................] duration: 1.8618516s FAIL (2025-08-13 03:11:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.861876939s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................X..X.......X...........X............] duration: 5.848762324s FAIL (2025-08-13 03:11:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.848790668s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.844209339s FAIL (2025-08-13 03:11:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.84424161s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................X..X.......X...........X...........] duration: 5.640654966s FAIL (2025-08-13 03:11:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.640686617s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X.......X..........................................] duration: 1.825610472s FAIL (2025-08-13 03:10:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.825638467s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.020513898s FAIL (2025-08-13 03:10:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.020547973s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.........................X..X.......X...........X..........] duration: 5.608546431s FAIL (2025-08-13 03:09:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.608568783s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X.......X.........................................] duration: 1.863382013s FAIL (2025-08-13 03:09:35 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.863407331s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.........................X..X.......X...........X.........] duration: 5.687724947s FAIL (2025-08-13 03:09:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.687747971s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.126822248s FAIL (2025-08-13 03:09:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.126850514s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.........................X..X.......X...........X........] duration: 5.657871737s FAIL (2025-08-13 03:09:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.657895793s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.872443942s FAIL (2025-08-13 03:09:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.87246769s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.861106839s FAIL (2025-08-13 03:08:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.861125355s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X.......X........................................] duration: 1.853324839s FAIL (2025-08-13 03:08:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.853350988s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X.........................X..X.......X...........X.......] duration: 5.694314362s FAIL (2025-08-13 03:08:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.694338709s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X.........................X..X.......X...........X......] duration: 5.618692795s FAIL (2025-08-13 03:07:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.61872079s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.897624189s FAIL (2025-08-13 03:07:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.89767697s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X.........................X..X.......X...........X.....] duration: 6.23700487s FAIL (2025-08-13 03:07:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 6.237023476s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X.......X.......................................] duration: 2.233289878s FAIL (2025-08-13 03:07:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.233320225s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000931216s FAIL (2025-08-13 03:07:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 30.000946025s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 7.621759623s FAIL (2025-08-13 03:07:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.621774904s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 20.016874897s FAIL (2025-08-13 03:06:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 20.016896068s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X.........................X..X.......X...........X....] duration: 6.083496826s FAIL (2025-08-13 03:06:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 6.083521053s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X.......X......................................] duration: 3.200830264s FAIL (2025-08-13 03:06:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 3.200875952s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 9.833509872s FAIL (2025-08-13 03:06:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.833528758s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X.........................X..X.......X...........X...] duration: 5.828733666s FAIL (2025-08-13 03:05:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.828759856s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [........X.........................X..X.......X...........X..] duration: 5.868609354s FAIL (2025-08-13 03:05:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.868639692s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 7.664002971s FAIL (2025-08-13 03:05:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.664026838s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X.......X.....................................] duration: 2.039335821s FAIL (2025-08-13 03:05:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.039361479s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........X.........................X..X.......X...........X.] duration: 5.769521453s FAIL (2025-08-13 03:04:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 5.769548734s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 7.212331155s FAIL (2025-08-13 03:04:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.212356874s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X.......X....................................] duration: 1.871155387s FAIL (2025-08-13 03:04:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.871186525s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 8.602897157s FAIL (2025-08-13 03:04:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.602921255s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X.........................X..X.......X...........X] duration: 6.134067821s FAIL (2025-08-13 03:03:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0587c93c696894d139905036976dc5a8de55bdefe20a18bf08fc8b3aad3ebc5b != before b1ac550f2e7c1e1aaa874e84017e78eb47189cdcb66c01bc3a4b326165815afc ======LOGS====== Aug 13 03:03:42.993513526 (105h37m37.424593657s) [upload=109957]: Starting upload Aug 13 03:03:42.993552260 (105h37m37.424632392s) [upload=109957]: Done starting upload Aug 13 03:03:42.993621643 (105h37m37.424701774s) [upload=109957]: Uploading segments... Aug 13 03:03:42.993934352 (105h37m37.425014483s) [upload=109957]: Got next segment. Inline: false Aug 13 03:03:42.993973416 (105h37m37.425053547s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:03:42.993977764 (105h37m37.425057895s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:03:43.443578956 (105h37m37.874659087s) [seg_pos={0 0}, upload=109957]: Begin upload segment... Aug 13 03:03:43.443632107 (105h37m37.874712238s) [seg_pos={0 0}, upload=109957]: Done begin upload segment. Aug 13 03:03:43.443653168 (105h37m37.874733299s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.443678086 (105h37m37.874758247s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444281681 (105h37m37.875361812s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444584290 (105h37m37.875664441s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444634746 (105h37m37.875714877s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.444911065 (105h37m37.875991196s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445131256 (105h37m37.876211387s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445161995 (105h37m37.876242126s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445351007 (105h37m37.876431138s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445569004 (105h37m37.876649125s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445649287 (105h37m37.876729428s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.445825144 (105h37m37.876905275s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446031067 (105h37m37.877111188s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446102775 (105h37m37.877182906s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446500956 (105h37m37.877581087s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.446854142 (105h37m37.877934273s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447086547 (105h37m37.878166678s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:43.447376121 (105h37m37.878456252s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Uploading piece... Aug 13 03:03:44.652348844 (105h37m39.083428965s) [upload=109957]: Next returned nil segment Aug 13 03:03:44.652371377 (105h37m39.083451498s) [upload=109957]: Waiting for error group managing segments... Aug 13 03:03:45.047287990 (105h37m39.478368111s) [noise=true , address=64.49.13.27:4967 , piece_id=UKI3VWNF3QYM3P5Q37UZGXRKVC73QWFP7PT6CZ5RFGQMRO42NGRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.405294633 (105h37m39.836374764s) [noise=true , address=104.36.82.39:4754 , piece_id=6SZLMVTDWE7HGQF2TTU667QMJCZQYKCSCSFROPAVHF45HM3VL2IQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.408247783 (105h37m39.839327914s) [noise=true , address=77.90.31.12:4593 , piece_id=RXTMTHOZ2WB25MDB5SOEARMT22GMJPBKHQZJ4FKSRD2FYQ7EPSFQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.412478458 (105h37m39.843558639s) [noise=true , address=104.36.82.36:4632 , piece_id=LEN7M24KXOKS3AGPF2AMIQNIS763WSPWZXXVAD2DN7N7U5MBECVQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.414116973 (105h37m39.845197104s) [noise=true , address=45.91.116.10:4096 , piece_id=A557GAUZZLG2QV2SFJCEOIVPA2CJHKPNTYBGHCBBL55FQGQ4NCBA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.419237570 (105h37m39.850317741s) [noise=true , address=104.36.82.43:4855 , piece_id=KVKNU2YXHW6BW6FYM4BMOBGJN7LVZGCGGOQ4WPJOWVURCC46DALQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.439790985 (105h37m39.870871116s) [noise=true , address=79.127.205.233:4629 , piece_id=HCBLZJUW6KRFGG7R5OYPTRFUNZWIIR7ONDLKDPOXKC642EYJBINA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.465333165 (105h37m39.896413296s) [noise=true , address=104.36.82.5:4858 , piece_id=NW6UFTJV45766AXHSG2L5XDD7VPQD6SSIAX3QYJW67G3A7WTXEZQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.488652273 (105h37m39.919732404s) [noise=true , address=104.36.82.19:4402 , piece_id=FHVJBJMKFENZ6MZ55BQYP55ABTI5LBHEJBSHZJM3OIHVHHESH74Q, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.501688806 (105h37m39.932768937s) [noise=true , address=104.36.82.45:4906 , piece_id=AAZTTLHH2X2FMAK5QHWN7CKY3E5KYJXQIQ47GYCB5RY5ZFTTFBRA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.578675336 (105h37m40.009755468s) [noise=true , address=104.36.82.23:4446 , piece_id=5NTASX7I7NXONP5HQBHZFGIRR6WACCPMQIF5MTG7EA4ZG4L74ROQ, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.586096966 (105h37m40.017177097s) [noise=true , address=104.36.82.16:4300 , piece_id=5KUYKRCIUZSNZ2DNXTYQYB33JVY6BCNPSELZAFECP4YENRXODX6A, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.587043938 (105h37m40.018124069s) [noise=true , address=77.90.31.19:4868 , piece_id=6C5WY3ZEXW4GBGUYZIZ7MF64EWNFJZWFOGFCXB5DCGI6TMY4KVUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602145251 (105h37m40.033225382s) [noise=true , address=104.36.82.23:4452 , piece_id=7YOCBDZQDVALDPEE2HE7VIABB75RPKSURBSDGUV76OXF66KI3TFA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: <nil> Aug 13 03:03:45.602173756 (105h37m40.033253887s) [seg_pos={0 0}, upload=109957]: Segment reached optimal threshold of 14 pieces. Aug 13 03:03:45.602431860 (105h37m40.033511991s) [noise=true , address=79.127.201.214:4316 , piece_id=LEDTESFFTBSP6MKTIUZHZNEJNEMWQBGY6JTMW2IJU3NSGPURVFJA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602455796 (105h37m40.033535927s) [noise=true , address=104.36.82.17:4358 , piece_id=QPRFXC22L23HR7QXENGSLOJNYYK6IT6GO673BYESUZ7TJLAREAMA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK): protocol: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48016->104.36.82.17:4358: use of closed network connection Aug 13 03:03:45.602507254 (105h37m40.033587385s) [noise=true , address=77.90.31.11:4570 , piece_id=7L5BOCWK2IE2XJKMQAOLKIYEQQVZNWG3XTRKZ253KUL2EAEFTFUA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602541300 (105h37m40.033621431s) [noise=true , address=104.36.82.44:4880 , piece_id=PXMXOEL7TDLT3MFFMMQVNJMLUZZXLRROEGYH54ON5FQ7T525ZOCA, seg_pos={0 0}, upload=109957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sFbwzYfjUGQqrKJhgwDg5dvormM1Xu4yc97awR5kgZUAsKiEn): protocol: expected piece hash; context canceled; EOF Aug 13 03:03:45.602572509 (105h37m40.03365263s) [seg_pos={0 0}, upload=109957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:03:45.602662792 (105h37m40.033742913s) [upload=109957]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:03:45.602667300 (105h37m40.033747431s) [upload=109957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:03:46.019530566 (105h37m40.450610697s) [upload=109957]: Done uploading segments... Aug 13 03:03:46.019550715 (105h37m40.450630846s) [upload=109957]: Upload finished. err: <nil> Duration: 6.134099212s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................X.......X...................................] duration: 1.85251942s FAIL (2025-08-13 03:03:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.852538667s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 8.06661829s FAIL (2025-08-13 03:03:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.066641315s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........X.........................X..X.......X...........] duration: 5.711530707s FAIL (2025-08-13 03:02:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5287e025b3cd3b0aefe8a79e22ae5e46805d492e522320960c8fb0254791163a != before 18ef631615bd58c58f6929f2a3a3256a662e46b10a1741658c110b75d45fec7d ======LOGS====== Aug 13 02:54:49.410532583 (105h28m43.841612714s) [upload=109802]: Starting upload Aug 13 02:54:49.410581436 (105h28m43.841661567s) [upload=109802]: Done starting upload Aug 13 02:54:49.410692278 (105h28m43.841772409s) [upload=109802]: Uploading segments... Aug 13 02:54:49.410799924 (105h28m43.841880085s) [upload=109802]: Got next segment. Inline: false Aug 13 02:54:49.410846072 (105h28m43.841926214s) [upload=109802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:54:49.410852545 (105h28m43.841932696s) [upload=109802]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:54:49.864437780 (105h28m44.295517911s) [seg_pos={0 0}, upload=109802]: Begin upload segment... Aug 13 02:54:49.864482857 (105h28m44.295562988s) [seg_pos={0 0}, upload=109802]: Done begin upload segment. Aug 13 02:54:49.864543442 (105h28m44.295623573s) [noise=true , address=77.90.31.15:4729 , piece_id=GD4HIUPV4OTT2TCNS2STVNSX2FKW45HX3XWYSLBNKTIPLOAHIQ7A, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.864605270 (105h28m44.295685441s) [noise=true , address=104.36.82.5:4845 , piece_id=3AZ2DURPGXWQUBTTGW3D6GMPULE5YGQTETS227R7KOSYJBQ4PLEA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.864941684 (105h28m44.296021805s) [noise=true , address=104.36.82.34:4579 , piece_id=RBAEQPAA5HEVU6EWL5H5ZS2C3KBMDAYHXVTN5DVEHC2ECMNCRGSQ, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.865069659 (105h28m44.29614979s) [noise=true , address=104.36.82.23:4461 , piece_id=YURWPKHC7XS442XBP5ZEVVAKKMNDWKUFU6OOJCYFAASNO4WUHLVA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.865224976 (105h28m44.296305107s) [noise=true , address=104.36.82.47:4530 , piece_id=NW7JOVDV2H4CGZ4ZYMX2KGEIQPXNIE6PKGLQYVGHJYJAQ7XBY7MA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.865513087 (105h28m44.296593208s) [noise=true , address=104.36.82.60:4366 , piece_id=25EMLXXCMPXGQ3BZOLB7E24DHOLIWPHNBEFNLYHEJEYQCRWFOJLA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.865761923 (105h28m44.296842054s) [noise=true , address=104.36.82.18:4387 , piece_id=YQCNWVPHHREHTKTCV7SL5QSGAYCYBXQOM6LERL33KK2HJ7WMAGZA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.866073871 (105h28m44.297154002s) [noise=true , address=104.36.82.19:4419 , piece_id=5KJJMUDRFR2A37IZIVUANMWRFMOZZK3IYT5C7LCKUDIFVTFNIWSA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.866329429 (105h28m44.29740956s) [noise=true , address=104.36.82.39:4750 , piece_id=SFDQIJAFNHASWBQNP5D6RN4RORSNL2ZWLUKWEX5NN3GUCHVDDQ7A, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.866571232 (105h28m44.297651363s) [noise=true , address=104.36.82.46:4494 , piece_id=X5FL6P5ZX62N2OKT5IR6CZVZR7J2K4G3M5A3XE5Q57QD2O44XRMA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.866825028 (105h28m44.297905159s) [noise=true , address=104.36.82.9:4028 , piece_id=EXBC3664SCW4NIA7TWJLMFIINTUWNOQ7QQE5IZSFRDPE37SH5QNQ, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.867037925 (105h28m44.298118056s) [noise=true , address=77.90.31.12:4621 , piece_id=CZZ3Y4PWYBHGTYDGPZSADGM2KUXCWI6KXJVXEY6LK3T5OZ54Y3MA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.867237126 (105h28m44.298317257s) [noise=true , address=104.36.82.62:4435 , piece_id=CXYSPKUGRYGFJCRHDSIOINSZXULLN6Z2XLBOMVG4MK4KDJWGVCZA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.867432420 (105h28m44.298512541s) [noise=true , address=104.36.82.43:4852 , piece_id=U7UTTJMV6TZRGVAYFWI6BBUDTQ3IVB37XF4XZ7S2DLURVDWSDPHQ, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.867636920 (105h28m44.298717052s) [noise=true , address=104.36.82.36:4621 , piece_id=4M7IYXO25UUHFCUZAJ6Q5XWC7QBK65L5PXNQX7PCAT6BCBZIAROA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.867831744 (105h28m44.298911865s) [noise=true , address=104.36.82.11:4125 , piece_id=HPRR533FKWCR6JXP742OAJMMXV3FNRUBTKRFRZGEKBRHJJOZDG2Q, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.868026747 (105h28m44.299106868s) [noise=true , address=104.36.82.58:4265 , piece_id=56PFHLHSQAEEZKZALYFCPJW4ZFSN4N5UAFX7LG3D5GVC5NI5SBRA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:49.868213754 (105h28m44.299293885s) [noise=true , address=64.49.13.140:4515 , piece_id=YHST6QB3UQTOC2G3NH3X2WDGXZ4LNKZRRKX66R6KB42YMJFQ7XKA, seg_pos={0 0}, upload=109802]: Uploading piece... Aug 13 02:54:51.042253784 (105h28m45.473333915s) [upload=109802]: Next returned nil segment Aug 13 02:54:51.042273020 (105h28m45.473353151s) [upload=109802]: Waiting for error group managing segments... Aug 13 02:54:51.340181215 (105h28m45.771261346s) [noise=true , address=64.49.13.140:4515 , piece_id=YHST6QB3UQTOC2G3NH3X2WDGXZ4LNKZRRKX66R6KB42YMJFQ7XKA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.841821285 (105h28m46.272901406s) [noise=true , address=104.36.82.5:4845 , piece_id=3AZ2DURPGXWQUBTTGW3D6GMPULE5YGQTETS227R7KOSYJBQ4PLEA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.856901098 (105h28m46.287981229s) [noise=true , address=104.36.82.60:4366 , piece_id=25EMLXXCMPXGQ3BZOLB7E24DHOLIWPHNBEFNLYHEJEYQCRWFOJLA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.857619863 (105h28m46.288699984s) [noise=true , address=104.36.82.58:4265 , piece_id=56PFHLHSQAEEZKZALYFCPJW4ZFSN4N5UAFX7LG3D5GVC5NI5SBRA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.859465063 (105h28m46.290545184s) [noise=true , address=104.36.82.18:4387 , piece_id=YQCNWVPHHREHTKTCV7SL5QSGAYCYBXQOM6LERL33KK2HJ7WMAGZA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.866614803 (105h28m46.297694934s) [noise=true , address=104.36.82.46:4494 , piece_id=X5FL6P5ZX62N2OKT5IR6CZVZR7J2K4G3M5A3XE5Q57QD2O44XRMA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.890262429 (105h28m46.32134259s) [noise=true , address=77.90.31.12:4621 , piece_id=CZZ3Y4PWYBHGTYDGPZSADGM2KUXCWI6KXJVXEY6LK3T5OZ54Y3MA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.891030158 (105h28m46.322110289s) [noise=true , address=104.36.82.19:4419 , piece_id=5KJJMUDRFR2A37IZIVUANMWRFMOZZK3IYT5C7LCKUDIFVTFNIWSA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.931891118 (105h28m46.362971249s) [noise=true , address=104.36.82.39:4750 , piece_id=SFDQIJAFNHASWBQNP5D6RN4RORSNL2ZWLUKWEX5NN3GUCHVDDQ7A, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.945310784 (105h28m46.376390915s) [noise=true , address=104.36.82.34:4579 , piece_id=RBAEQPAA5HEVU6EWL5H5ZS2C3KBMDAYHXVTN5DVEHC2ECMNCRGSQ, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.957947522 (105h28m46.389027653s) [noise=true , address=104.36.82.36:4621 , piece_id=4M7IYXO25UUHFCUZAJ6Q5XWC7QBK65L5PXNQX7PCAT6BCBZIAROA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:51.964733436 (105h28m46.395813567s) [noise=true , address=104.36.82.9:4028 , piece_id=EXBC3664SCW4NIA7TWJLMFIINTUWNOQ7QQE5IZSFRDPE37SH5QNQ, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:52.520080243 (105h28m46.951160364s) [noise=true , address=104.36.82.62:4435 , piece_id=CXYSPKUGRYGFJCRHDSIOINSZXULLN6Z2XLBOMVG4MK4KDJWGVCZA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:52.641998082 (105h28m47.073078203s) [noise=true , address=104.36.82.11:4125 , piece_id=HPRR533FKWCR6JXP742OAJMMXV3FNRUBTKRFRZGEKBRHJJOZDG2Q, seg_pos={0 0}, upload=109802]: Done uploading piece. err: <nil> Aug 13 02:54:52.642026426 (105h28m47.073106547s) [seg_pos={0 0}, upload=109802]: Segment reached optimal threshold of 14 pieces. Aug 13 02:54:52.642149312 (105h28m47.073229443s) [noise=true , address=104.36.82.23:4461 , piece_id=YURWPKHC7XS442XBP5ZEVVAKKMNDWKUFU6OOJCYFAASNO4WUHLVA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fNrbSfErEQRnruJCJAVvkoNwQSPWnvNtxunHn6zrQegnNAkxA): protocol: expected piece hash; context canceled; EOF Aug 13 02:54:52.642199026 (105h28m47.073279157s) [noise=true , address=104.36.82.47:4530 , piece_id=NW7JOVDV2H4CGZ4ZYMX2KGEIQPXNIE6PKGLQYVGHJYJAQ7XBY7MA, seg_pos={0 0}, upload=109802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WVxMFx9jm6CT7GcXg2qgq3Be9UPFf1KZKssqufSk2ebdBBmxR): protocol: expected piece hash; context canceled; EOF Aug 13 02:54:52.642251046 (105h28m47.073331227s) [noise=true , address=104.36.82.43:4852 , piece_id=U7UTTJMV6TZRGVAYFWI6BBUDTQ3IVB37XF4XZ7S2DLURVDWSDPHQ, seg_pos={0 0}, upload=109802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CWXaz1RJdY28Y4jCshs5Ti4GhJKR1LJL1pBBEWGWrzbfZXfav): protocol: expected piece hash; context canceled; EOF Aug 13 02:54:52.642377037 (105h28m47.073457168s) [noise=true , address=77.90.31.15:4729 , piece_id=GD4HIUPV4OTT2TCNS2STVNSX2FKW45HX3XWYSLBNKTIPLOAHIQ7A, seg_pos={0 0}, upload=109802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ArQdwHzHSm7c2QZkTt1kQsfzocvCNG2D9yoQgnmbS7HYLCXhy): protocol: expected piece hash; context canceled; EOF Aug 13 02:54:52.642418666 (105h28m47.073498787s) [seg_pos={0 0}, upload=109802]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:54:52.642506224 (105h28m47.073586356s) [upload=109802]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:54:52.642510753 (105h28m47.073590884s) [upload=109802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:54:52.929221442 (105h28m47.360301573s) [upload=109802]: Done uploading segments... Aug 13 02:54:52.929249466 (105h28m47.360329607s) [upload=109802]: Upload finished. err: <nil> Duration: 5.711566645s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)