Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [......X....X........X.......................X...............] duration: 1.810618209s FAIL (2025-08-13 10:42:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.810644568s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................X...X......X.............X............] duration: 5.816566313s FAIL (2025-08-13 10:42:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 5.816597062s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.......X....X........X.......................X..............] duration: 1.844092929s FAIL (2025-08-13 10:42:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.844124018s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.866496128s FAIL (2025-08-13 10:41:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.866521957s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [........X....X........X.......................X.............] duration: 1.867264703s FAIL (2025-08-13 10:41:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.86728855s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................X...X......X.............X...........] duration: 5.876699713s FAIL (2025-08-13 10:41:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 5.876726363s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.158862499s FAIL (2025-08-13 10:41:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.15888973s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................X...X......X.............X..........] duration: 5.684789173s FAIL (2025-08-13 10:41:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 5.684816928s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.........X....X........X.......................X............] duration: 1.859231625s FAIL (2025-08-13 10:40:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.859254257s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.820105039s FAIL (2025-08-13 10:40:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.820128223s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................X...X......X.............X.........] duration: 5.547553032s FAIL (2025-08-13 10:40:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 5.547585023s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.211047744s FAIL (2025-08-13 10:40:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.211073434s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................X...X......X.............X........] duration: 5.745682625s FAIL (2025-08-13 10:40:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 5.745718785s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........X....X........X.......................X...........] duration: 1.987782788s FAIL (2025-08-13 10:39:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.987809218s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.159522604s FAIL (2025-08-13 10:39:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.159710665s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................X...X......X.............X.......] duration: 5.782442149s FAIL (2025-08-13 10:39:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 5.782476995s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................X...X......X.............X......] duration: 8.050007242s FAIL (2025-08-13 10:39:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 8.050031969s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................X...X......X.............X.....] duration: 5.781166617s FAIL (2025-08-13 10:38:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 5.781187015s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...........X....X........X.......................X..........] duration: 1.970644245s FAIL (2025-08-13 10:38:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.97067238s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.222019813s FAIL (2025-08-13 10:38:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.222097713s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................X...X......X.............X....] duration: 5.796599433s FAIL (2025-08-13 10:37:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 5.796626775s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.356703465s FAIL (2025-08-13 10:37:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.356723955s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [............X....X........X.......................X.........] duration: 2.104863612s FAIL (2025-08-13 10:37:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.104889343s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................X...X......X.............X...] duration: 8.260802449s FAIL (2025-08-13 10:37:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 8.260821856s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.226842947s FAIL (2025-08-13 10:36:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.226868947s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................X...X......X.............X..] duration: 9.410681273s FAIL (2025-08-13 10:36:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 9.410709216s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............X....X........X.......................X........] duration: 2.136918988s FAIL (2025-08-13 10:36:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.136943124s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.173165993s FAIL (2025-08-13 10:35:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.173187725s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................X...X......X.............X.] duration: 5.760667144s FAIL (2025-08-13 10:35:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 5.760695147s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..............X....X........X.......................X.......] duration: 1.831184292s FAIL (2025-08-13 10:35:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.831210353s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.826804669s FAIL (2025-08-13 10:35:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.826822924s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...............X....X........X.......................X......] duration: 1.874800576s FAIL (2025-08-13 10:35:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.874864389s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................X...X......X.............X] duration: 5.946329914s FAIL (2025-08-13 10:34:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9d6cf9118fe23ca506e4d60e6035c741dd2da17880ff2b41d55926bd25555049 != before b9f11ef995e0eeeef484d496eed30b654a1d289e23262e3856ac9ca7c42c1c37 ======LOGS====== Aug 13 10:34:46.190287505 (113h8m40.621367636s) [upload=117761]: Starting upload Aug 13 10:34:46.190354564 (113h8m40.621434695s) [upload=117761]: Done starting upload Aug 13 10:34:46.190421682 (113h8m40.621501803s) [upload=117761]: Uploading segments... Aug 13 10:34:46.190524619 (113h8m40.621604751s) [upload=117761]: Got next segment. Inline: false Aug 13 10:34:46.190565026 (113h8m40.621645157s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:34:46.190568844 (113h8m40.621648976s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:34:46.635963699 (113h8m41.06704383s) [seg_pos={0 0}, upload=117761]: Begin upload segment... Aug 13 10:34:46.636025358 (113h8m41.067105479s) [seg_pos={0 0}, upload=117761]: Done begin upload segment. Aug 13 10:34:46.636048071 (113h8m41.067128192s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636164574 (113h8m41.067244725s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636632890 (113h8m41.067713011s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636871958 (113h8m41.067952089s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.636923125 (113h8m41.068003246s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637333991 (113h8m41.068414112s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637358257 (113h8m41.068438389s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637643193 (113h8m41.068723324s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637701535 (113h8m41.068781656s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.637923930 (113h8m41.069004061s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638080891 (113h8m41.069161022s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638181733 (113h8m41.069261854s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638375333 (113h8m41.069455464s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638498239 (113h8m41.06957837s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638624370 (113h8m41.069704491s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.638871553 (113h8m41.069951684s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639258754 (113h8m41.070338885s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:46.639680291 (113h8m41.070760422s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Uploading piece... Aug 13 10:34:47.791865208 (113h8m42.222945339s) [upload=117761]: Next returned nil segment Aug 13 10:34:47.791880367 (113h8m42.222960498s) [upload=117761]: Waiting for error group managing segments... Aug 13 10:34:48.102502615 (113h8m42.533582736s) [noise=true , address=64.49.13.140:4510 , piece_id=ARBADSU6SGHUEK37Z5Y3VKVIAAGLGRMCAUQCZ2QT47CTILMWFLMA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.102715392 (113h8m42.533795523s) [noise=true , address=66.23.193.175:4557 , piece_id=SCENFW2K5E3OQQ2X33HD7LLZLTX3YQK4VCIEN4J4O4RLMGUI5RLQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.482098393 (113h8m42.913178524s) [noise=true , address=79.127.220.101:4071 , piece_id=J2UZDFKYKBW6LKIXKCU2BNCH7IYWPHZW2NKJBXV25I7EUD7RHNUA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.501681181 (113h8m42.932761322s) [noise=true , address=79.127.219.48:4075 , piece_id=7VWBAFARAZ3WLXA6PRRYJNXTNL7S7KGVQ6A5N6UO4DIHFCD3XN6Q, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.571639596 (113h8m43.002719727s) [noise=true , address=77.90.31.14:4693 , piece_id=QILKSYYXG5KNA7ZOJQXNV5GYNENJ2IFLFQWSQXFKNNP5GRZOCQYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.585200993 (113h8m43.016281124s) [noise=true , address=45.91.116.9:4266 , piece_id=WIKXKCSAXVPYSMMBTRULOJ3OYPOKPYI6YBFDYX2EFN6XATVV2Q7A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.608388278 (113h8m43.039468409s) [noise=true , address=104.36.82.37:4678 , piece_id=FVATRHQRRPJRKFDGQOD5GWBF2SL2FCE36FHJ3CLXYZ3RIN7SI7UQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.619651039 (113h8m43.05073116s) [noise=true , address=79.127.201.215:4359 , piece_id=FKK52KUM4PPYAIGAU4J75PZKMNVOMGWOHTU67QSNWC6PS5NL7BEQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.620652605 (113h8m43.051732736s) [noise=true , address=77.90.31.15:4716 , piece_id=YF6R4DYRP6IUO2B4SVD3P5GVXGWUPKAJBEC6AQKJYFUATZHD2UGQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.650590336 (113h8m43.081670467s) [noise=true , address=104.36.82.23:4462 , piece_id=XPCTIP46AT4CNRXFILMVA2XLX3SY2PXANTH36DUXFWD46CKIAH3A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.676740510 (113h8m43.107820641s) [noise=true , address=104.36.82.4:4805 , piece_id=ERR4EFC22L2NET6XUY6L5USFZU32SPTK67B5OUK6R4PRHTN3IEOA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.699623063 (113h8m43.130703194s) [noise=true , address=104.36.82.34:4553 , piece_id=B3AD5XT7VHUDYPZ6JTIF6TTNJXJTX2SRL7FU2ZQOYDU4UVTRFQMQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.709417953 (113h8m43.140498084s) [noise=true , address=77.90.31.11:4575 , piece_id=MVUGBS7RDPZM2GKSZCEZZF6KSVTNPOXDZXB4752LVTHKUOGKOAUQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781735202 (113h8m43.212815333s) [noise=true , address=104.36.82.5:4848 , piece_id=LXBY6B7PKDLB6BSA6YCJCNRYEQO65A6PSQPILEOYCXFJO5OVHTHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: <nil> Aug 13 10:34:48.781754849 (113h8m43.21283498s) [seg_pos={0 0}, upload=117761]: Segment reached optimal threshold of 14 pieces. Aug 13 10:34:48.781871332 (113h8m43.212951453s) [noise=true , address=104.36.82.60:4372 , piece_id=STIQ4AIQH2XFKGSUPWV7MVBHEZUGOYWCHQTAGIKMEWPOH7X6JPYQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781921709 (113h8m43.21300184s) [noise=true , address=104.36.82.58:4291 , piece_id=HDSI7PYQWXWHCGJJTOG5ELJB2AUJ3KNN3CIAP64BH4LQCUKYUCHQ, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.781951185 (113h8m43.213031316s) [noise=true , address=104.36.82.44:4882 , piece_id=PYWOPHRXPGHQ2V4I3OBWABCS6CWX2KEY7NZPJYFWSSG52ONDFG6A, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782000449 (113h8m43.21308058s) [noise=true , address=104.36.82.62:4441 , piece_id=ENNUORH2RUPQF4NSFJRKIWL7G3IZ65E3O4HCGUCHK5XN2ZCAEQHA, seg_pos={0 0}, upload=117761]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 13 10:34:48.782014215 (113h8m43.213094346s) [seg_pos={0 0}, upload=117761]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:34:48.782068970 (113h8m43.213149101s) [upload=117761]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:34:48.782073178 (113h8m43.213153309s) [upload=117761]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:34:49.073737612 (113h8m43.504817743s) [upload=117761]: Done uploading segments... Aug 13 10:34:49.073757671 (113h8m43.504837802s) [upload=117761]: Upload finished. err: <nil> Duration: 5.946357436s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.272233111s FAIL (2025-08-13 10:34:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.272257799s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [................X....X........X.......................X.....] duration: 2.029873038s FAIL (2025-08-13 10:33:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.029897053s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................X...X......X.............] duration: 5.650420076s FAIL (2025-08-13 10:33:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after d26652193dfb9cb87448fd367541112b440819ee683a57e9f9c29a99735e6f1f != before 9ff9d86dfd655098c5b9ead4a76d88b7108995a8c5e8cd2ce381c4337ca9af9d ======LOGS====== Aug 13 10:23:52.445348879 (112h57m46.87642901s) [upload=117574]: Starting upload Aug 13 10:23:52.445395027 (112h57m46.876475158s) [upload=117574]: Done starting upload Aug 13 10:23:52.445480712 (112h57m46.876560843s) [upload=117574]: Uploading segments... Aug 13 10:23:52.445878152 (112h57m46.876958283s) [upload=117574]: Got next segment. Inline: false Aug 13 10:23:52.446085379 (112h57m46.8771655s) [upload=117574]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:23:52.446093223 (112h57m46.877173354s) [upload=117574]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:23:52.901576721 (112h57m47.332656853s) [seg_pos={0 0}, upload=117574]: Begin upload segment... Aug 13 10:23:52.901638079 (112h57m47.33271821s) [noise=true , address=104.36.82.62:4439 , piece_id=BZPRHLLZXM2B7XU3JDHIKB244ZGJHJLMF5QBQWRA75ZELAG6KXLA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.901704306 (112h57m47.332784467s) [noise=true , address=77.90.31.13:4645 , piece_id=JPZWPBGMVVBDWAYUBQUZ4EY3UNZZ42FNPHXOFKTATKS4AYHATVQA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902012335 (112h57m47.333092466s) [seg_pos={0 0}, upload=117574]: Done begin upload segment. Aug 13 10:23:52.902195095 (112h57m47.333275216s) [noise=true , address=77.90.31.19:4866 , piece_id=2D4WVDUWEBV2DZCC4OPBUJ5O7MKFFCFQDR54JDTEURJ35W43KNRA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902451535 (112h57m47.333531666s) [noise=true , address=104.36.82.36:4625 , piece_id=4TXL3QCRRRRC24IW2KMUWGX6E3XLHW72EHYO4S6GZNFSO5ZKAUYA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902728355 (112h57m47.333808486s) [noise=true , address=64.49.13.26:4946 , piece_id=KCITGQFUGIPGURKN2TMD3XIKUL5EBLW5ENDCB44RJOWHNCAK32GQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902971751 (112h57m47.334051882s) [noise=true , address=104.36.82.23:4475 , piece_id=XFM3FXWTQMQ66MV4BWCU6G2GGHKUVWCX7ISUNPIFCIOB5N6G2H3Q, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903217822 (112h57m47.334297953s) [noise=true , address=104.36.82.34:4575 , piece_id=6XN72EW5FREZFOTH43JSYIP24USSAGESJQSEXH3I2HMU6YVB3URA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903337280 (112h57m47.334417411s) [noise=true , address=104.36.82.44:4887 , piece_id=TNWOAH6SREWAQD6HCZ2ADSM6QNXJ2QDCJTYN65FS6A7OZCV226EA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903575536 (112h57m47.334655667s) [noise=true , address=79.127.219.38:4013 , piece_id=ZNLAQDSOGUQGJT4C3CHYJ34WGYZOXVVJFZX6R4PWUCBX6X6BFFSA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903802550 (112h57m47.334882681s) [noise=true , address=64.49.13.27:4967 , piece_id=VN3YZVFR3XZVH55FW7KACPYAGKX4P24GSLCKE4SCRDP632CQ4DWA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903957818 (112h57m47.335037949s) [noise=true , address=104.36.82.60:4355 , piece_id=2VS7BNMYMH25ILXP5FIMEVMV4QZZIHZ3FPXWGY2FXPAUJ437OKXA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904300984 (112h57m47.335381105s) [noise=true , address=66.23.193.175:4555 , piece_id=6R3RONZXILUEPHEN2M6QWPXXKDJQ4GEFC6LOZ4NJK7VZBB5RGDDQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904791583 (112h57m47.335871724s) [noise=true , address=104.36.82.5:4848 , piece_id=GMOJUKKVPJQCMHLIZRRGV5UAEHZQS6FTHFPOF5CA6LB2GKRYFQQQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904989150 (112h57m47.336069301s) [noise=true , address=77.90.31.15:4722 , piece_id=LD3KGVZ7XWPSKUUJFRXPJIS5SR2M44HTHG5IC67R65R3XOATX3FA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905479449 (112h57m47.33655959s) [noise=true , address=77.90.31.12:4612 , piece_id=S6FFDZXWWCVMUJ5BOKDGL6VR2MCGNGEDUXBFX3A5JZ4B3YH5L2QQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905607584 (112h57m47.336687715s) [noise=true , address=66.23.193.175:4554 , piece_id=TYNNCBAJH4BGFXLOEARL2OFJ2PO6TDYUZDDREZT4PYA6CVQVNKBQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905935552 (112h57m47.337015683s) [noise=true , address=104.36.82.37:4676 , piece_id=HXL3V3XZX3DG27V5D5D3BGUPJBAEWZXSEU62TLZT7KY2G57VB7BA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.906289920 (112h57m47.337370051s) [noise=true , address=104.36.82.3:4771 , piece_id=LJNUI2CZGCLD4ZEQRJU53SO2Z5HIU2TACPPGYAJV5DYMRPHAOKXA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:54.029149259 (112h57m48.46022939s) [upload=117574]: Next returned nil segment Aug 13 10:23:54.029171981 (112h57m48.460252102s) [upload=117574]: Waiting for error group managing segments... Aug 13 10:23:54.319286411 (112h57m48.750366542s) [noise=true , address=66.23.193.175:4555 , piece_id=6R3RONZXILUEPHEN2M6QWPXXKDJQ4GEFC6LOZ4NJK7VZBB5RGDDQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.335770941 (112h57m48.766851072s) [noise=true , address=66.23.193.175:4554 , piece_id=TYNNCBAJH4BGFXLOEARL2OFJ2PO6TDYUZDDREZT4PYA6CVQVNKBQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.339234699 (112h57m48.77031483s) [noise=true , address=64.49.13.27:4967 , piece_id=VN3YZVFR3XZVH55FW7KACPYAGKX4P24GSLCKE4SCRDP632CQ4DWA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.825170732 (112h57m49.256250893s) [noise=true , address=79.127.219.38:4013 , piece_id=ZNLAQDSOGUQGJT4C3CHYJ34WGYZOXVVJFZX6R4PWUCBX6X6BFFSA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.847491389 (112h57m49.278571521s) [noise=true , address=77.90.31.13:4645 , piece_id=JPZWPBGMVVBDWAYUBQUZ4EY3UNZZ42FNPHXOFKTATKS4AYHATVQA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.854857724 (112h57m49.285937855s) [noise=true , address=77.90.31.15:4722 , piece_id=LD3KGVZ7XWPSKUUJFRXPJIS5SR2M44HTHG5IC67R65R3XOATX3FA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.870014554 (112h57m49.301094685s) [noise=true , address=104.36.82.34:4575 , piece_id=6XN72EW5FREZFOTH43JSYIP24USSAGESJQSEXH3I2HMU6YVB3URA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.870974931 (112h57m49.302055053s) [noise=true , address=104.36.82.23:4475 , piece_id=XFM3FXWTQMQ66MV4BWCU6G2GGHKUVWCX7ISUNPIFCIOB5N6G2H3Q, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.873543647 (112h57m49.304623778s) [noise=true , address=77.90.31.12:4612 , piece_id=S6FFDZXWWCVMUJ5BOKDGL6VR2MCGNGEDUXBFX3A5JZ4B3YH5L2QQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.909057627 (112h57m49.340137768s) [noise=true , address=104.36.82.37:4676 , piece_id=HXL3V3XZX3DG27V5D5D3BGUPJBAEWZXSEU62TLZT7KY2G57VB7BA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.909713933 (112h57m49.340794064s) [noise=true , address=104.36.82.60:4355 , piece_id=2VS7BNMYMH25ILXP5FIMEVMV4QZZIHZ3FPXWGY2FXPAUJ437OKXA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.937892618 (112h57m49.368972739s) [noise=true , address=77.90.31.19:4866 , piece_id=2D4WVDUWEBV2DZCC4OPBUJ5O7MKFFCFQDR54JDTEURJ35W43KNRA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.963464456 (112h57m49.394544587s) [noise=true , address=104.36.82.62:4439 , piece_id=BZPRHLLZXM2B7XU3JDHIKB244ZGJHJLMF5QBQWRA75ZELAG6KXLA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.980942818 (112h57m49.412022949s) [noise=true , address=104.36.82.36:4625 , piece_id=4TXL3QCRRRRC24IW2KMUWGX6E3XLHW72EHYO4S6GZNFSO5ZKAUYA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.980971663 (112h57m49.412051794s) [seg_pos={0 0}, upload=117574]: Segment reached optimal threshold of 14 pieces. Aug 13 10:23:54.981084308 (112h57m49.412164439s) [noise=true , address=64.49.13.26:4946 , piece_id=KCITGQFUGIPGURKN2TMD3XIKUL5EBLW5ENDCB44RJOWHNCAK32GQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hdE48UnygY1dCRASR39FkumY2V8f5Xb68Pd1WJ4AMLUnnGAfT): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981140025 (112h57m49.412220156s) [noise=true , address=104.36.82.5:4848 , piece_id=GMOJUKKVPJQCMHLIZRRGV5UAEHZQS6FTHFPOF5CA6LB2GKRYFQQQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xvod7N8eEwunLDYWAvVUP1Cp1CdEr3quJUMG5kWyBrwzzr9vp): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981245527 (112h57m49.412325658s) [noise=true , address=104.36.82.44:4887 , piece_id=TNWOAH6SREWAQD6HCZ2ADSM6QNXJ2QDCJTYN65FS6A7OZCV226EA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981288219 (112h57m49.41236836s) [noise=true , address=104.36.82.3:4771 , piece_id=LJNUI2CZGCLD4ZEQRJU53SO2Z5HIU2TACPPGYAJV5DYMRPHAOKXA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rcgrMxg2fRfvh7EA1eUehzD5rSH1krUd5W5eC8KTo7vjYWFLL): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981355768 (112h57m49.412435899s) [seg_pos={0 0}, upload=117574]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:23:54.981457813 (112h57m49.412537944s) [upload=117574]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:23:54.981464075 (112h57m49.412544216s) [upload=117574]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:23:55.278135040 (112h57m49.709215171s) [upload=117574]: Done uploading segments... Aug 13 10:23:55.278152384 (112h57m49.709232505s) [upload=117574]: Upload finished. err: <nil> Duration: 5.650450795s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.23313318s FAIL (2025-08-13 10:33:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.23315441s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.78720704s FAIL (2025-08-13 10:33:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.787227748s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................X...X......X............] duration: 6.049749533s FAIL (2025-08-13 10:32:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after d26652193dfb9cb87448fd367541112b440819ee683a57e9f9c29a99735e6f1f != before 9ff9d86dfd655098c5b9ead4a76d88b7108995a8c5e8cd2ce381c4337ca9af9d ======LOGS====== Aug 13 10:23:52.445348879 (112h57m46.87642901s) [upload=117574]: Starting upload Aug 13 10:23:52.445395027 (112h57m46.876475158s) [upload=117574]: Done starting upload Aug 13 10:23:52.445480712 (112h57m46.876560843s) [upload=117574]: Uploading segments... Aug 13 10:23:52.445878152 (112h57m46.876958283s) [upload=117574]: Got next segment. Inline: false Aug 13 10:23:52.446085379 (112h57m46.8771655s) [upload=117574]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:23:52.446093223 (112h57m46.877173354s) [upload=117574]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:23:52.901576721 (112h57m47.332656853s) [seg_pos={0 0}, upload=117574]: Begin upload segment... Aug 13 10:23:52.901638079 (112h57m47.33271821s) [noise=true , address=104.36.82.62:4439 , piece_id=BZPRHLLZXM2B7XU3JDHIKB244ZGJHJLMF5QBQWRA75ZELAG6KXLA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.901704306 (112h57m47.332784467s) [noise=true , address=77.90.31.13:4645 , piece_id=JPZWPBGMVVBDWAYUBQUZ4EY3UNZZ42FNPHXOFKTATKS4AYHATVQA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902012335 (112h57m47.333092466s) [seg_pos={0 0}, upload=117574]: Done begin upload segment. Aug 13 10:23:52.902195095 (112h57m47.333275216s) [noise=true , address=77.90.31.19:4866 , piece_id=2D4WVDUWEBV2DZCC4OPBUJ5O7MKFFCFQDR54JDTEURJ35W43KNRA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902451535 (112h57m47.333531666s) [noise=true , address=104.36.82.36:4625 , piece_id=4TXL3QCRRRRC24IW2KMUWGX6E3XLHW72EHYO4S6GZNFSO5ZKAUYA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902728355 (112h57m47.333808486s) [noise=true , address=64.49.13.26:4946 , piece_id=KCITGQFUGIPGURKN2TMD3XIKUL5EBLW5ENDCB44RJOWHNCAK32GQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902971751 (112h57m47.334051882s) [noise=true , address=104.36.82.23:4475 , piece_id=XFM3FXWTQMQ66MV4BWCU6G2GGHKUVWCX7ISUNPIFCIOB5N6G2H3Q, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903217822 (112h57m47.334297953s) [noise=true , address=104.36.82.34:4575 , piece_id=6XN72EW5FREZFOTH43JSYIP24USSAGESJQSEXH3I2HMU6YVB3URA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903337280 (112h57m47.334417411s) [noise=true , address=104.36.82.44:4887 , piece_id=TNWOAH6SREWAQD6HCZ2ADSM6QNXJ2QDCJTYN65FS6A7OZCV226EA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903575536 (112h57m47.334655667s) [noise=true , address=79.127.219.38:4013 , piece_id=ZNLAQDSOGUQGJT4C3CHYJ34WGYZOXVVJFZX6R4PWUCBX6X6BFFSA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903802550 (112h57m47.334882681s) [noise=true , address=64.49.13.27:4967 , piece_id=VN3YZVFR3XZVH55FW7KACPYAGKX4P24GSLCKE4SCRDP632CQ4DWA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903957818 (112h57m47.335037949s) [noise=true , address=104.36.82.60:4355 , piece_id=2VS7BNMYMH25ILXP5FIMEVMV4QZZIHZ3FPXWGY2FXPAUJ437OKXA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904300984 (112h57m47.335381105s) [noise=true , address=66.23.193.175:4555 , piece_id=6R3RONZXILUEPHEN2M6QWPXXKDJQ4GEFC6LOZ4NJK7VZBB5RGDDQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904791583 (112h57m47.335871724s) [noise=true , address=104.36.82.5:4848 , piece_id=GMOJUKKVPJQCMHLIZRRGV5UAEHZQS6FTHFPOF5CA6LB2GKRYFQQQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904989150 (112h57m47.336069301s) [noise=true , address=77.90.31.15:4722 , piece_id=LD3KGVZ7XWPSKUUJFRXPJIS5SR2M44HTHG5IC67R65R3XOATX3FA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905479449 (112h57m47.33655959s) [noise=true , address=77.90.31.12:4612 , piece_id=S6FFDZXWWCVMUJ5BOKDGL6VR2MCGNGEDUXBFX3A5JZ4B3YH5L2QQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905607584 (112h57m47.336687715s) [noise=true , address=66.23.193.175:4554 , piece_id=TYNNCBAJH4BGFXLOEARL2OFJ2PO6TDYUZDDREZT4PYA6CVQVNKBQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905935552 (112h57m47.337015683s) [noise=true , address=104.36.82.37:4676 , piece_id=HXL3V3XZX3DG27V5D5D3BGUPJBAEWZXSEU62TLZT7KY2G57VB7BA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.906289920 (112h57m47.337370051s) [noise=true , address=104.36.82.3:4771 , piece_id=LJNUI2CZGCLD4ZEQRJU53SO2Z5HIU2TACPPGYAJV5DYMRPHAOKXA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:54.029149259 (112h57m48.46022939s) [upload=117574]: Next returned nil segment Aug 13 10:23:54.029171981 (112h57m48.460252102s) [upload=117574]: Waiting for error group managing segments... Aug 13 10:23:54.319286411 (112h57m48.750366542s) [noise=true , address=66.23.193.175:4555 , piece_id=6R3RONZXILUEPHEN2M6QWPXXKDJQ4GEFC6LOZ4NJK7VZBB5RGDDQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.335770941 (112h57m48.766851072s) [noise=true , address=66.23.193.175:4554 , piece_id=TYNNCBAJH4BGFXLOEARL2OFJ2PO6TDYUZDDREZT4PYA6CVQVNKBQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.339234699 (112h57m48.77031483s) [noise=true , address=64.49.13.27:4967 , piece_id=VN3YZVFR3XZVH55FW7KACPYAGKX4P24GSLCKE4SCRDP632CQ4DWA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.825170732 (112h57m49.256250893s) [noise=true , address=79.127.219.38:4013 , piece_id=ZNLAQDSOGUQGJT4C3CHYJ34WGYZOXVVJFZX6R4PWUCBX6X6BFFSA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.847491389 (112h57m49.278571521s) [noise=true , address=77.90.31.13:4645 , piece_id=JPZWPBGMVVBDWAYUBQUZ4EY3UNZZ42FNPHXOFKTATKS4AYHATVQA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.854857724 (112h57m49.285937855s) [noise=true , address=77.90.31.15:4722 , piece_id=LD3KGVZ7XWPSKUUJFRXPJIS5SR2M44HTHG5IC67R65R3XOATX3FA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.870014554 (112h57m49.301094685s) [noise=true , address=104.36.82.34:4575 , piece_id=6XN72EW5FREZFOTH43JSYIP24USSAGESJQSEXH3I2HMU6YVB3URA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.870974931 (112h57m49.302055053s) [noise=true , address=104.36.82.23:4475 , piece_id=XFM3FXWTQMQ66MV4BWCU6G2GGHKUVWCX7ISUNPIFCIOB5N6G2H3Q, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.873543647 (112h57m49.304623778s) [noise=true , address=77.90.31.12:4612 , piece_id=S6FFDZXWWCVMUJ5BOKDGL6VR2MCGNGEDUXBFX3A5JZ4B3YH5L2QQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.909057627 (112h57m49.340137768s) [noise=true , address=104.36.82.37:4676 , piece_id=HXL3V3XZX3DG27V5D5D3BGUPJBAEWZXSEU62TLZT7KY2G57VB7BA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.909713933 (112h57m49.340794064s) [noise=true , address=104.36.82.60:4355 , piece_id=2VS7BNMYMH25ILXP5FIMEVMV4QZZIHZ3FPXWGY2FXPAUJ437OKXA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.937892618 (112h57m49.368972739s) [noise=true , address=77.90.31.19:4866 , piece_id=2D4WVDUWEBV2DZCC4OPBUJ5O7MKFFCFQDR54JDTEURJ35W43KNRA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.963464456 (112h57m49.394544587s) [noise=true , address=104.36.82.62:4439 , piece_id=BZPRHLLZXM2B7XU3JDHIKB244ZGJHJLMF5QBQWRA75ZELAG6KXLA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.980942818 (112h57m49.412022949s) [noise=true , address=104.36.82.36:4625 , piece_id=4TXL3QCRRRRC24IW2KMUWGX6E3XLHW72EHYO4S6GZNFSO5ZKAUYA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.980971663 (112h57m49.412051794s) [seg_pos={0 0}, upload=117574]: Segment reached optimal threshold of 14 pieces. Aug 13 10:23:54.981084308 (112h57m49.412164439s) [noise=true , address=64.49.13.26:4946 , piece_id=KCITGQFUGIPGURKN2TMD3XIKUL5EBLW5ENDCB44RJOWHNCAK32GQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hdE48UnygY1dCRASR39FkumY2V8f5Xb68Pd1WJ4AMLUnnGAfT): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981140025 (112h57m49.412220156s) [noise=true , address=104.36.82.5:4848 , piece_id=GMOJUKKVPJQCMHLIZRRGV5UAEHZQS6FTHFPOF5CA6LB2GKRYFQQQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xvod7N8eEwunLDYWAvVUP1Cp1CdEr3quJUMG5kWyBrwzzr9vp): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981245527 (112h57m49.412325658s) [noise=true , address=104.36.82.44:4887 , piece_id=TNWOAH6SREWAQD6HCZ2ADSM6QNXJ2QDCJTYN65FS6A7OZCV226EA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981288219 (112h57m49.41236836s) [noise=true , address=104.36.82.3:4771 , piece_id=LJNUI2CZGCLD4ZEQRJU53SO2Z5HIU2TACPPGYAJV5DYMRPHAOKXA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rcgrMxg2fRfvh7EA1eUehzD5rSH1krUd5W5eC8KTo7vjYWFLL): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981355768 (112h57m49.412435899s) [seg_pos={0 0}, upload=117574]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:23:54.981457813 (112h57m49.412537944s) [upload=117574]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:23:54.981464075 (112h57m49.412544216s) [upload=117574]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:23:55.278135040 (112h57m49.709215171s) [upload=117574]: Done uploading segments... Aug 13 10:23:55.278152384 (112h57m49.709232505s) [upload=117574]: Upload finished. err: <nil> Duration: 6.049777837s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.................X....X........X.......................X....] duration: 2.085811238s FAIL (2025-08-13 10:32:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.085852866s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..................X....X........X.......................X...] duration: 2.15967788s FAIL (2025-08-13 10:32:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.159727194s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.908274503s FAIL (2025-08-13 10:31:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.908296595s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................X...X......X...........] duration: 8.600501877s FAIL (2025-08-13 10:31:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after d26652193dfb9cb87448fd367541112b440819ee683a57e9f9c29a99735e6f1f != before 9ff9d86dfd655098c5b9ead4a76d88b7108995a8c5e8cd2ce381c4337ca9af9d ======LOGS====== Aug 13 10:23:52.445348879 (112h57m46.87642901s) [upload=117574]: Starting upload Aug 13 10:23:52.445395027 (112h57m46.876475158s) [upload=117574]: Done starting upload Aug 13 10:23:52.445480712 (112h57m46.876560843s) [upload=117574]: Uploading segments... Aug 13 10:23:52.445878152 (112h57m46.876958283s) [upload=117574]: Got next segment. Inline: false Aug 13 10:23:52.446085379 (112h57m46.8771655s) [upload=117574]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:23:52.446093223 (112h57m46.877173354s) [upload=117574]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:23:52.901576721 (112h57m47.332656853s) [seg_pos={0 0}, upload=117574]: Begin upload segment... Aug 13 10:23:52.901638079 (112h57m47.33271821s) [noise=true , address=104.36.82.62:4439 , piece_id=BZPRHLLZXM2B7XU3JDHIKB244ZGJHJLMF5QBQWRA75ZELAG6KXLA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.901704306 (112h57m47.332784467s) [noise=true , address=77.90.31.13:4645 , piece_id=JPZWPBGMVVBDWAYUBQUZ4EY3UNZZ42FNPHXOFKTATKS4AYHATVQA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902012335 (112h57m47.333092466s) [seg_pos={0 0}, upload=117574]: Done begin upload segment. Aug 13 10:23:52.902195095 (112h57m47.333275216s) [noise=true , address=77.90.31.19:4866 , piece_id=2D4WVDUWEBV2DZCC4OPBUJ5O7MKFFCFQDR54JDTEURJ35W43KNRA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902451535 (112h57m47.333531666s) [noise=true , address=104.36.82.36:4625 , piece_id=4TXL3QCRRRRC24IW2KMUWGX6E3XLHW72EHYO4S6GZNFSO5ZKAUYA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902728355 (112h57m47.333808486s) [noise=true , address=64.49.13.26:4946 , piece_id=KCITGQFUGIPGURKN2TMD3XIKUL5EBLW5ENDCB44RJOWHNCAK32GQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902971751 (112h57m47.334051882s) [noise=true , address=104.36.82.23:4475 , piece_id=XFM3FXWTQMQ66MV4BWCU6G2GGHKUVWCX7ISUNPIFCIOB5N6G2H3Q, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903217822 (112h57m47.334297953s) [noise=true , address=104.36.82.34:4575 , piece_id=6XN72EW5FREZFOTH43JSYIP24USSAGESJQSEXH3I2HMU6YVB3URA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903337280 (112h57m47.334417411s) [noise=true , address=104.36.82.44:4887 , piece_id=TNWOAH6SREWAQD6HCZ2ADSM6QNXJ2QDCJTYN65FS6A7OZCV226EA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903575536 (112h57m47.334655667s) [noise=true , address=79.127.219.38:4013 , piece_id=ZNLAQDSOGUQGJT4C3CHYJ34WGYZOXVVJFZX6R4PWUCBX6X6BFFSA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903802550 (112h57m47.334882681s) [noise=true , address=64.49.13.27:4967 , piece_id=VN3YZVFR3XZVH55FW7KACPYAGKX4P24GSLCKE4SCRDP632CQ4DWA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903957818 (112h57m47.335037949s) [noise=true , address=104.36.82.60:4355 , piece_id=2VS7BNMYMH25ILXP5FIMEVMV4QZZIHZ3FPXWGY2FXPAUJ437OKXA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904300984 (112h57m47.335381105s) [noise=true , address=66.23.193.175:4555 , piece_id=6R3RONZXILUEPHEN2M6QWPXXKDJQ4GEFC6LOZ4NJK7VZBB5RGDDQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904791583 (112h57m47.335871724s) [noise=true , address=104.36.82.5:4848 , piece_id=GMOJUKKVPJQCMHLIZRRGV5UAEHZQS6FTHFPOF5CA6LB2GKRYFQQQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904989150 (112h57m47.336069301s) [noise=true , address=77.90.31.15:4722 , piece_id=LD3KGVZ7XWPSKUUJFRXPJIS5SR2M44HTHG5IC67R65R3XOATX3FA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905479449 (112h57m47.33655959s) [noise=true , address=77.90.31.12:4612 , piece_id=S6FFDZXWWCVMUJ5BOKDGL6VR2MCGNGEDUXBFX3A5JZ4B3YH5L2QQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905607584 (112h57m47.336687715s) [noise=true , address=66.23.193.175:4554 , piece_id=TYNNCBAJH4BGFXLOEARL2OFJ2PO6TDYUZDDREZT4PYA6CVQVNKBQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905935552 (112h57m47.337015683s) [noise=true , address=104.36.82.37:4676 , piece_id=HXL3V3XZX3DG27V5D5D3BGUPJBAEWZXSEU62TLZT7KY2G57VB7BA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.906289920 (112h57m47.337370051s) [noise=true , address=104.36.82.3:4771 , piece_id=LJNUI2CZGCLD4ZEQRJU53SO2Z5HIU2TACPPGYAJV5DYMRPHAOKXA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:54.029149259 (112h57m48.46022939s) [upload=117574]: Next returned nil segment Aug 13 10:23:54.029171981 (112h57m48.460252102s) [upload=117574]: Waiting for error group managing segments... Aug 13 10:23:54.319286411 (112h57m48.750366542s) [noise=true , address=66.23.193.175:4555 , piece_id=6R3RONZXILUEPHEN2M6QWPXXKDJQ4GEFC6LOZ4NJK7VZBB5RGDDQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.335770941 (112h57m48.766851072s) [noise=true , address=66.23.193.175:4554 , piece_id=TYNNCBAJH4BGFXLOEARL2OFJ2PO6TDYUZDDREZT4PYA6CVQVNKBQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.339234699 (112h57m48.77031483s) [noise=true , address=64.49.13.27:4967 , piece_id=VN3YZVFR3XZVH55FW7KACPYAGKX4P24GSLCKE4SCRDP632CQ4DWA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.825170732 (112h57m49.256250893s) [noise=true , address=79.127.219.38:4013 , piece_id=ZNLAQDSOGUQGJT4C3CHYJ34WGYZOXVVJFZX6R4PWUCBX6X6BFFSA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.847491389 (112h57m49.278571521s) [noise=true , address=77.90.31.13:4645 , piece_id=JPZWPBGMVVBDWAYUBQUZ4EY3UNZZ42FNPHXOFKTATKS4AYHATVQA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.854857724 (112h57m49.285937855s) [noise=true , address=77.90.31.15:4722 , piece_id=LD3KGVZ7XWPSKUUJFRXPJIS5SR2M44HTHG5IC67R65R3XOATX3FA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.870014554 (112h57m49.301094685s) [noise=true , address=104.36.82.34:4575 , piece_id=6XN72EW5FREZFOTH43JSYIP24USSAGESJQSEXH3I2HMU6YVB3URA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.870974931 (112h57m49.302055053s) [noise=true , address=104.36.82.23:4475 , piece_id=XFM3FXWTQMQ66MV4BWCU6G2GGHKUVWCX7ISUNPIFCIOB5N6G2H3Q, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.873543647 (112h57m49.304623778s) [noise=true , address=77.90.31.12:4612 , piece_id=S6FFDZXWWCVMUJ5BOKDGL6VR2MCGNGEDUXBFX3A5JZ4B3YH5L2QQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.909057627 (112h57m49.340137768s) [noise=true , address=104.36.82.37:4676 , piece_id=HXL3V3XZX3DG27V5D5D3BGUPJBAEWZXSEU62TLZT7KY2G57VB7BA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.909713933 (112h57m49.340794064s) [noise=true , address=104.36.82.60:4355 , piece_id=2VS7BNMYMH25ILXP5FIMEVMV4QZZIHZ3FPXWGY2FXPAUJ437OKXA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.937892618 (112h57m49.368972739s) [noise=true , address=77.90.31.19:4866 , piece_id=2D4WVDUWEBV2DZCC4OPBUJ5O7MKFFCFQDR54JDTEURJ35W43KNRA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.963464456 (112h57m49.394544587s) [noise=true , address=104.36.82.62:4439 , piece_id=BZPRHLLZXM2B7XU3JDHIKB244ZGJHJLMF5QBQWRA75ZELAG6KXLA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.980942818 (112h57m49.412022949s) [noise=true , address=104.36.82.36:4625 , piece_id=4TXL3QCRRRRC24IW2KMUWGX6E3XLHW72EHYO4S6GZNFSO5ZKAUYA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.980971663 (112h57m49.412051794s) [seg_pos={0 0}, upload=117574]: Segment reached optimal threshold of 14 pieces. Aug 13 10:23:54.981084308 (112h57m49.412164439s) [noise=true , address=64.49.13.26:4946 , piece_id=KCITGQFUGIPGURKN2TMD3XIKUL5EBLW5ENDCB44RJOWHNCAK32GQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hdE48UnygY1dCRASR39FkumY2V8f5Xb68Pd1WJ4AMLUnnGAfT): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981140025 (112h57m49.412220156s) [noise=true , address=104.36.82.5:4848 , piece_id=GMOJUKKVPJQCMHLIZRRGV5UAEHZQS6FTHFPOF5CA6LB2GKRYFQQQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xvod7N8eEwunLDYWAvVUP1Cp1CdEr3quJUMG5kWyBrwzzr9vp): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981245527 (112h57m49.412325658s) [noise=true , address=104.36.82.44:4887 , piece_id=TNWOAH6SREWAQD6HCZ2ADSM6QNXJ2QDCJTYN65FS6A7OZCV226EA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981288219 (112h57m49.41236836s) [noise=true , address=104.36.82.3:4771 , piece_id=LJNUI2CZGCLD4ZEQRJU53SO2Z5HIU2TACPPGYAJV5DYMRPHAOKXA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rcgrMxg2fRfvh7EA1eUehzD5rSH1krUd5W5eC8KTo7vjYWFLL): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981355768 (112h57m49.412435899s) [seg_pos={0 0}, upload=117574]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:23:54.981457813 (112h57m49.412537944s) [upload=117574]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:23:54.981464075 (112h57m49.412544216s) [upload=117574]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:23:55.278135040 (112h57m49.709215171s) [upload=117574]: Done uploading segments... Aug 13 10:23:55.278152384 (112h57m49.709232505s) [upload=117574]: Upload finished. err: <nil> Duration: 8.600525212s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...................X....X........X.......................X..] duration: 2.095130186s FAIL (2025-08-13 10:31:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.09515833s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................X...X......X..........] duration: 6.034767899s FAIL (2025-08-13 10:31:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after d26652193dfb9cb87448fd367541112b440819ee683a57e9f9c29a99735e6f1f != before 9ff9d86dfd655098c5b9ead4a76d88b7108995a8c5e8cd2ce381c4337ca9af9d ======LOGS====== Aug 13 10:23:52.445348879 (112h57m46.87642901s) [upload=117574]: Starting upload Aug 13 10:23:52.445395027 (112h57m46.876475158s) [upload=117574]: Done starting upload Aug 13 10:23:52.445480712 (112h57m46.876560843s) [upload=117574]: Uploading segments... Aug 13 10:23:52.445878152 (112h57m46.876958283s) [upload=117574]: Got next segment. Inline: false Aug 13 10:23:52.446085379 (112h57m46.8771655s) [upload=117574]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:23:52.446093223 (112h57m46.877173354s) [upload=117574]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:23:52.901576721 (112h57m47.332656853s) [seg_pos={0 0}, upload=117574]: Begin upload segment... Aug 13 10:23:52.901638079 (112h57m47.33271821s) [noise=true , address=104.36.82.62:4439 , piece_id=BZPRHLLZXM2B7XU3JDHIKB244ZGJHJLMF5QBQWRA75ZELAG6KXLA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.901704306 (112h57m47.332784467s) [noise=true , address=77.90.31.13:4645 , piece_id=JPZWPBGMVVBDWAYUBQUZ4EY3UNZZ42FNPHXOFKTATKS4AYHATVQA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902012335 (112h57m47.333092466s) [seg_pos={0 0}, upload=117574]: Done begin upload segment. Aug 13 10:23:52.902195095 (112h57m47.333275216s) [noise=true , address=77.90.31.19:4866 , piece_id=2D4WVDUWEBV2DZCC4OPBUJ5O7MKFFCFQDR54JDTEURJ35W43KNRA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902451535 (112h57m47.333531666s) [noise=true , address=104.36.82.36:4625 , piece_id=4TXL3QCRRRRC24IW2KMUWGX6E3XLHW72EHYO4S6GZNFSO5ZKAUYA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902728355 (112h57m47.333808486s) [noise=true , address=64.49.13.26:4946 , piece_id=KCITGQFUGIPGURKN2TMD3XIKUL5EBLW5ENDCB44RJOWHNCAK32GQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902971751 (112h57m47.334051882s) [noise=true , address=104.36.82.23:4475 , piece_id=XFM3FXWTQMQ66MV4BWCU6G2GGHKUVWCX7ISUNPIFCIOB5N6G2H3Q, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903217822 (112h57m47.334297953s) [noise=true , address=104.36.82.34:4575 , piece_id=6XN72EW5FREZFOTH43JSYIP24USSAGESJQSEXH3I2HMU6YVB3URA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903337280 (112h57m47.334417411s) [noise=true , address=104.36.82.44:4887 , piece_id=TNWOAH6SREWAQD6HCZ2ADSM6QNXJ2QDCJTYN65FS6A7OZCV226EA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903575536 (112h57m47.334655667s) [noise=true , address=79.127.219.38:4013 , piece_id=ZNLAQDSOGUQGJT4C3CHYJ34WGYZOXVVJFZX6R4PWUCBX6X6BFFSA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903802550 (112h57m47.334882681s) [noise=true , address=64.49.13.27:4967 , piece_id=VN3YZVFR3XZVH55FW7KACPYAGKX4P24GSLCKE4SCRDP632CQ4DWA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903957818 (112h57m47.335037949s) [noise=true , address=104.36.82.60:4355 , piece_id=2VS7BNMYMH25ILXP5FIMEVMV4QZZIHZ3FPXWGY2FXPAUJ437OKXA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904300984 (112h57m47.335381105s) [noise=true , address=66.23.193.175:4555 , piece_id=6R3RONZXILUEPHEN2M6QWPXXKDJQ4GEFC6LOZ4NJK7VZBB5RGDDQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904791583 (112h57m47.335871724s) [noise=true , address=104.36.82.5:4848 , piece_id=GMOJUKKVPJQCMHLIZRRGV5UAEHZQS6FTHFPOF5CA6LB2GKRYFQQQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904989150 (112h57m47.336069301s) [noise=true , address=77.90.31.15:4722 , piece_id=LD3KGVZ7XWPSKUUJFRXPJIS5SR2M44HTHG5IC67R65R3XOATX3FA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905479449 (112h57m47.33655959s) [noise=true , address=77.90.31.12:4612 , piece_id=S6FFDZXWWCVMUJ5BOKDGL6VR2MCGNGEDUXBFX3A5JZ4B3YH5L2QQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905607584 (112h57m47.336687715s) [noise=true , address=66.23.193.175:4554 , piece_id=TYNNCBAJH4BGFXLOEARL2OFJ2PO6TDYUZDDREZT4PYA6CVQVNKBQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905935552 (112h57m47.337015683s) [noise=true , address=104.36.82.37:4676 , piece_id=HXL3V3XZX3DG27V5D5D3BGUPJBAEWZXSEU62TLZT7KY2G57VB7BA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.906289920 (112h57m47.337370051s) [noise=true , address=104.36.82.3:4771 , piece_id=LJNUI2CZGCLD4ZEQRJU53SO2Z5HIU2TACPPGYAJV5DYMRPHAOKXA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:54.029149259 (112h57m48.46022939s) [upload=117574]: Next returned nil segment Aug 13 10:23:54.029171981 (112h57m48.460252102s) [upload=117574]: Waiting for error group managing segments... Aug 13 10:23:54.319286411 (112h57m48.750366542s) [noise=true , address=66.23.193.175:4555 , piece_id=6R3RONZXILUEPHEN2M6QWPXXKDJQ4GEFC6LOZ4NJK7VZBB5RGDDQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.335770941 (112h57m48.766851072s) [noise=true , address=66.23.193.175:4554 , piece_id=TYNNCBAJH4BGFXLOEARL2OFJ2PO6TDYUZDDREZT4PYA6CVQVNKBQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.339234699 (112h57m48.77031483s) [noise=true , address=64.49.13.27:4967 , piece_id=VN3YZVFR3XZVH55FW7KACPYAGKX4P24GSLCKE4SCRDP632CQ4DWA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.825170732 (112h57m49.256250893s) [noise=true , address=79.127.219.38:4013 , piece_id=ZNLAQDSOGUQGJT4C3CHYJ34WGYZOXVVJFZX6R4PWUCBX6X6BFFSA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.847491389 (112h57m49.278571521s) [noise=true , address=77.90.31.13:4645 , piece_id=JPZWPBGMVVBDWAYUBQUZ4EY3UNZZ42FNPHXOFKTATKS4AYHATVQA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.854857724 (112h57m49.285937855s) [noise=true , address=77.90.31.15:4722 , piece_id=LD3KGVZ7XWPSKUUJFRXPJIS5SR2M44HTHG5IC67R65R3XOATX3FA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.870014554 (112h57m49.301094685s) [noise=true , address=104.36.82.34:4575 , piece_id=6XN72EW5FREZFOTH43JSYIP24USSAGESJQSEXH3I2HMU6YVB3URA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.870974931 (112h57m49.302055053s) [noise=true , address=104.36.82.23:4475 , piece_id=XFM3FXWTQMQ66MV4BWCU6G2GGHKUVWCX7ISUNPIFCIOB5N6G2H3Q, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.873543647 (112h57m49.304623778s) [noise=true , address=77.90.31.12:4612 , piece_id=S6FFDZXWWCVMUJ5BOKDGL6VR2MCGNGEDUXBFX3A5JZ4B3YH5L2QQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.909057627 (112h57m49.340137768s) [noise=true , address=104.36.82.37:4676 , piece_id=HXL3V3XZX3DG27V5D5D3BGUPJBAEWZXSEU62TLZT7KY2G57VB7BA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.909713933 (112h57m49.340794064s) [noise=true , address=104.36.82.60:4355 , piece_id=2VS7BNMYMH25ILXP5FIMEVMV4QZZIHZ3FPXWGY2FXPAUJ437OKXA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.937892618 (112h57m49.368972739s) [noise=true , address=77.90.31.19:4866 , piece_id=2D4WVDUWEBV2DZCC4OPBUJ5O7MKFFCFQDR54JDTEURJ35W43KNRA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.963464456 (112h57m49.394544587s) [noise=true , address=104.36.82.62:4439 , piece_id=BZPRHLLZXM2B7XU3JDHIKB244ZGJHJLMF5QBQWRA75ZELAG6KXLA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.980942818 (112h57m49.412022949s) [noise=true , address=104.36.82.36:4625 , piece_id=4TXL3QCRRRRC24IW2KMUWGX6E3XLHW72EHYO4S6GZNFSO5ZKAUYA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.980971663 (112h57m49.412051794s) [seg_pos={0 0}, upload=117574]: Segment reached optimal threshold of 14 pieces. Aug 13 10:23:54.981084308 (112h57m49.412164439s) [noise=true , address=64.49.13.26:4946 , piece_id=KCITGQFUGIPGURKN2TMD3XIKUL5EBLW5ENDCB44RJOWHNCAK32GQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hdE48UnygY1dCRASR39FkumY2V8f5Xb68Pd1WJ4AMLUnnGAfT): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981140025 (112h57m49.412220156s) [noise=true , address=104.36.82.5:4848 , piece_id=GMOJUKKVPJQCMHLIZRRGV5UAEHZQS6FTHFPOF5CA6LB2GKRYFQQQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xvod7N8eEwunLDYWAvVUP1Cp1CdEr3quJUMG5kWyBrwzzr9vp): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981245527 (112h57m49.412325658s) [noise=true , address=104.36.82.44:4887 , piece_id=TNWOAH6SREWAQD6HCZ2ADSM6QNXJ2QDCJTYN65FS6A7OZCV226EA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981288219 (112h57m49.41236836s) [noise=true , address=104.36.82.3:4771 , piece_id=LJNUI2CZGCLD4ZEQRJU53SO2Z5HIU2TACPPGYAJV5DYMRPHAOKXA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rcgrMxg2fRfvh7EA1eUehzD5rSH1krUd5W5eC8KTo7vjYWFLL): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981355768 (112h57m49.412435899s) [seg_pos={0 0}, upload=117574]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:23:54.981457813 (112h57m49.412537944s) [upload=117574]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:23:54.981464075 (112h57m49.412544216s) [upload=117574]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:23:55.278135040 (112h57m49.709215171s) [upload=117574]: Done uploading segments... Aug 13 10:23:55.278152384 (112h57m49.709232505s) [upload=117574]: Upload finished. err: <nil> Duration: 6.034784832s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [....................X....X........X.......................X.] duration: 2.023624062s FAIL (2025-08-13 10:31:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.023647979s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.038904924s FAIL (2025-08-13 10:30:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.038925873s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.....................X....X........X.......................X] duration: 1.051605955s FAIL (2025-08-13 10:30:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 10:30:30.126234689 (113h4m24.55731482s) [upload=117690]: Starting upload Aug 13 10:30:30.126285686 (113h4m24.557365807s) [upload=117690]: Done starting upload Aug 13 10:30:30.126363775 (113h4m24.557443906s) [upload=117690]: Uploading segments... Aug 13 10:30:30.126406818 (113h4m24.557486949s) [upload=117690]: Got next segment. Inline: true Aug 13 10:30:30.126410946 (113h4m24.557491077s) [upload=117690]: Waiting for error group managing segments... Aug 13 10:30:30.126449639 (113h4m24.55752977s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:30:30.126453668 (113h4m24.557533799s) [upload=117690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 10:30:30.126456373 (113h4m24.557536504s) [upload=117690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:30:30.386069844 (113h4m24.817149975s) [upload=117690]: Done uploading segments... Aug 13 10:30:30.386103788 (113h4m24.817183919s) [upload=117690]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.051633047s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................X...X......X.........] duration: 6.036662765s FAIL (2025-08-13 10:30:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after d26652193dfb9cb87448fd367541112b440819ee683a57e9f9c29a99735e6f1f != before 9ff9d86dfd655098c5b9ead4a76d88b7108995a8c5e8cd2ce381c4337ca9af9d ======LOGS====== Aug 13 10:23:52.445348879 (112h57m46.87642901s) [upload=117574]: Starting upload Aug 13 10:23:52.445395027 (112h57m46.876475158s) [upload=117574]: Done starting upload Aug 13 10:23:52.445480712 (112h57m46.876560843s) [upload=117574]: Uploading segments... Aug 13 10:23:52.445878152 (112h57m46.876958283s) [upload=117574]: Got next segment. Inline: false Aug 13 10:23:52.446085379 (112h57m46.8771655s) [upload=117574]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 10:23:52.446093223 (112h57m46.877173354s) [upload=117574]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 10:23:52.901576721 (112h57m47.332656853s) [seg_pos={0 0}, upload=117574]: Begin upload segment... Aug 13 10:23:52.901638079 (112h57m47.33271821s) [noise=true , address=104.36.82.62:4439 , piece_id=BZPRHLLZXM2B7XU3JDHIKB244ZGJHJLMF5QBQWRA75ZELAG6KXLA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.901704306 (112h57m47.332784467s) [noise=true , address=77.90.31.13:4645 , piece_id=JPZWPBGMVVBDWAYUBQUZ4EY3UNZZ42FNPHXOFKTATKS4AYHATVQA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902012335 (112h57m47.333092466s) [seg_pos={0 0}, upload=117574]: Done begin upload segment. Aug 13 10:23:52.902195095 (112h57m47.333275216s) [noise=true , address=77.90.31.19:4866 , piece_id=2D4WVDUWEBV2DZCC4OPBUJ5O7MKFFCFQDR54JDTEURJ35W43KNRA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902451535 (112h57m47.333531666s) [noise=true , address=104.36.82.36:4625 , piece_id=4TXL3QCRRRRC24IW2KMUWGX6E3XLHW72EHYO4S6GZNFSO5ZKAUYA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902728355 (112h57m47.333808486s) [noise=true , address=64.49.13.26:4946 , piece_id=KCITGQFUGIPGURKN2TMD3XIKUL5EBLW5ENDCB44RJOWHNCAK32GQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.902971751 (112h57m47.334051882s) [noise=true , address=104.36.82.23:4475 , piece_id=XFM3FXWTQMQ66MV4BWCU6G2GGHKUVWCX7ISUNPIFCIOB5N6G2H3Q, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903217822 (112h57m47.334297953s) [noise=true , address=104.36.82.34:4575 , piece_id=6XN72EW5FREZFOTH43JSYIP24USSAGESJQSEXH3I2HMU6YVB3URA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903337280 (112h57m47.334417411s) [noise=true , address=104.36.82.44:4887 , piece_id=TNWOAH6SREWAQD6HCZ2ADSM6QNXJ2QDCJTYN65FS6A7OZCV226EA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903575536 (112h57m47.334655667s) [noise=true , address=79.127.219.38:4013 , piece_id=ZNLAQDSOGUQGJT4C3CHYJ34WGYZOXVVJFZX6R4PWUCBX6X6BFFSA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903802550 (112h57m47.334882681s) [noise=true , address=64.49.13.27:4967 , piece_id=VN3YZVFR3XZVH55FW7KACPYAGKX4P24GSLCKE4SCRDP632CQ4DWA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.903957818 (112h57m47.335037949s) [noise=true , address=104.36.82.60:4355 , piece_id=2VS7BNMYMH25ILXP5FIMEVMV4QZZIHZ3FPXWGY2FXPAUJ437OKXA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904300984 (112h57m47.335381105s) [noise=true , address=66.23.193.175:4555 , piece_id=6R3RONZXILUEPHEN2M6QWPXXKDJQ4GEFC6LOZ4NJK7VZBB5RGDDQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904791583 (112h57m47.335871724s) [noise=true , address=104.36.82.5:4848 , piece_id=GMOJUKKVPJQCMHLIZRRGV5UAEHZQS6FTHFPOF5CA6LB2GKRYFQQQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.904989150 (112h57m47.336069301s) [noise=true , address=77.90.31.15:4722 , piece_id=LD3KGVZ7XWPSKUUJFRXPJIS5SR2M44HTHG5IC67R65R3XOATX3FA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905479449 (112h57m47.33655959s) [noise=true , address=77.90.31.12:4612 , piece_id=S6FFDZXWWCVMUJ5BOKDGL6VR2MCGNGEDUXBFX3A5JZ4B3YH5L2QQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905607584 (112h57m47.336687715s) [noise=true , address=66.23.193.175:4554 , piece_id=TYNNCBAJH4BGFXLOEARL2OFJ2PO6TDYUZDDREZT4PYA6CVQVNKBQ, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.905935552 (112h57m47.337015683s) [noise=true , address=104.36.82.37:4676 , piece_id=HXL3V3XZX3DG27V5D5D3BGUPJBAEWZXSEU62TLZT7KY2G57VB7BA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:52.906289920 (112h57m47.337370051s) [noise=true , address=104.36.82.3:4771 , piece_id=LJNUI2CZGCLD4ZEQRJU53SO2Z5HIU2TACPPGYAJV5DYMRPHAOKXA, seg_pos={0 0}, upload=117574]: Uploading piece... Aug 13 10:23:54.029149259 (112h57m48.46022939s) [upload=117574]: Next returned nil segment Aug 13 10:23:54.029171981 (112h57m48.460252102s) [upload=117574]: Waiting for error group managing segments... Aug 13 10:23:54.319286411 (112h57m48.750366542s) [noise=true , address=66.23.193.175:4555 , piece_id=6R3RONZXILUEPHEN2M6QWPXXKDJQ4GEFC6LOZ4NJK7VZBB5RGDDQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.335770941 (112h57m48.766851072s) [noise=true , address=66.23.193.175:4554 , piece_id=TYNNCBAJH4BGFXLOEARL2OFJ2PO6TDYUZDDREZT4PYA6CVQVNKBQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.339234699 (112h57m48.77031483s) [noise=true , address=64.49.13.27:4967 , piece_id=VN3YZVFR3XZVH55FW7KACPYAGKX4P24GSLCKE4SCRDP632CQ4DWA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.825170732 (112h57m49.256250893s) [noise=true , address=79.127.219.38:4013 , piece_id=ZNLAQDSOGUQGJT4C3CHYJ34WGYZOXVVJFZX6R4PWUCBX6X6BFFSA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.847491389 (112h57m49.278571521s) [noise=true , address=77.90.31.13:4645 , piece_id=JPZWPBGMVVBDWAYUBQUZ4EY3UNZZ42FNPHXOFKTATKS4AYHATVQA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.854857724 (112h57m49.285937855s) [noise=true , address=77.90.31.15:4722 , piece_id=LD3KGVZ7XWPSKUUJFRXPJIS5SR2M44HTHG5IC67R65R3XOATX3FA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.870014554 (112h57m49.301094685s) [noise=true , address=104.36.82.34:4575 , piece_id=6XN72EW5FREZFOTH43JSYIP24USSAGESJQSEXH3I2HMU6YVB3URA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.870974931 (112h57m49.302055053s) [noise=true , address=104.36.82.23:4475 , piece_id=XFM3FXWTQMQ66MV4BWCU6G2GGHKUVWCX7ISUNPIFCIOB5N6G2H3Q, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.873543647 (112h57m49.304623778s) [noise=true , address=77.90.31.12:4612 , piece_id=S6FFDZXWWCVMUJ5BOKDGL6VR2MCGNGEDUXBFX3A5JZ4B3YH5L2QQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.909057627 (112h57m49.340137768s) [noise=true , address=104.36.82.37:4676 , piece_id=HXL3V3XZX3DG27V5D5D3BGUPJBAEWZXSEU62TLZT7KY2G57VB7BA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.909713933 (112h57m49.340794064s) [noise=true , address=104.36.82.60:4355 , piece_id=2VS7BNMYMH25ILXP5FIMEVMV4QZZIHZ3FPXWGY2FXPAUJ437OKXA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.937892618 (112h57m49.368972739s) [noise=true , address=77.90.31.19:4866 , piece_id=2D4WVDUWEBV2DZCC4OPBUJ5O7MKFFCFQDR54JDTEURJ35W43KNRA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.963464456 (112h57m49.394544587s) [noise=true , address=104.36.82.62:4439 , piece_id=BZPRHLLZXM2B7XU3JDHIKB244ZGJHJLMF5QBQWRA75ZELAG6KXLA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.980942818 (112h57m49.412022949s) [noise=true , address=104.36.82.36:4625 , piece_id=4TXL3QCRRRRC24IW2KMUWGX6E3XLHW72EHYO4S6GZNFSO5ZKAUYA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: <nil> Aug 13 10:23:54.980971663 (112h57m49.412051794s) [seg_pos={0 0}, upload=117574]: Segment reached optimal threshold of 14 pieces. Aug 13 10:23:54.981084308 (112h57m49.412164439s) [noise=true , address=64.49.13.26:4946 , piece_id=KCITGQFUGIPGURKN2TMD3XIKUL5EBLW5ENDCB44RJOWHNCAK32GQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hdE48UnygY1dCRASR39FkumY2V8f5Xb68Pd1WJ4AMLUnnGAfT): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981140025 (112h57m49.412220156s) [noise=true , address=104.36.82.5:4848 , piece_id=GMOJUKKVPJQCMHLIZRRGV5UAEHZQS6FTHFPOF5CA6LB2GKRYFQQQ, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xvod7N8eEwunLDYWAvVUP1Cp1CdEr3quJUMG5kWyBrwzzr9vp): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981245527 (112h57m49.412325658s) [noise=true , address=104.36.82.44:4887 , piece_id=TNWOAH6SREWAQD6HCZ2ADSM6QNXJ2QDCJTYN65FS6A7OZCV226EA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981288219 (112h57m49.41236836s) [noise=true , address=104.36.82.3:4771 , piece_id=LJNUI2CZGCLD4ZEQRJU53SO2Z5HIU2TACPPGYAJV5DYMRPHAOKXA, seg_pos={0 0}, upload=117574]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rcgrMxg2fRfvh7EA1eUehzD5rSH1krUd5W5eC8KTo7vjYWFLL): protocol: expected piece hash; context canceled; EOF Aug 13 10:23:54.981355768 (112h57m49.412435899s) [seg_pos={0 0}, upload=117574]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 10:23:54.981457813 (112h57m49.412537944s) [upload=117574]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 10:23:54.981464075 (112h57m49.412544216s) [upload=117574]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 10:23:55.278135040 (112h57m49.709215171s) [upload=117574]: Done uploading segments... Aug 13 10:23:55.278152384 (112h57m49.709232505s) [upload=117574]: Upload finished. err: <nil> Duration: 6.036687762s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.233869623s FAIL (2025-08-13 10:30:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.233902756s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)