Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 896005 reports (Page 15539 of 17921)

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [................................................XXXXXXXX.XXX] duration: 30.000870867s FAIL (2025-08-14 22:34:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000890845s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.628021496s FAIL (2025-08-14 22:34:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.628045181s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 8.825583032s FAIL (2025-08-14 22:33:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.825612276s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................X.........X.X...............X.........] duration: 7.026556225s FAIL (2025-08-14 22:33:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 7.026581603s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [..X..................X......X................X......X.......] duration: 1.958321459s FAIL (2025-08-14 22:33:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 14 22:28:15.058033232 (149h2m9.489113363s) [upload=155017]: Starting upload Aug 14 22:28:15.058069361 (149h2m9.489149482s) [upload=155017]: Done starting upload Aug 14 22:28:15.058119347 (149h2m9.489199478s) [upload=155017]: Uploading segments... Aug 14 22:28:15.058162900 (149h2m9.489243021s) [upload=155017]: Got next segment. Inline: true Aug 14 22:28:15.058166457 (149h2m9.489246588s) [upload=155017]: Waiting for error group managing segments... Aug 14 22:28:15.058197747 (149h2m9.489277878s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:28:15.058201274 (149h2m9.489281405s) [upload=155017]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:28:15.058204019 (149h2m9.48928414s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:28:16.900184965 (149h2m11.331265086s) [upload=155017]: Done uploading segments... Aug 14 22:28:16.900202930 (149h2m11.331283061s) [upload=155017]: Upload finished. err: <nil> Duration:* 1.958349923s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.................................................XXXXXXXX.XX] duration: 30.000483561s FAIL (2025-08-14 22:33:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000497547s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................X.........X.X...............X........] duration: 6.925788678s FAIL (2025-08-14 22:33:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.925809638s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 12.567171092s FAIL (2025-08-14 22:32:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.567184518s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..................................................XXXXXXXX.X] duration: 30.001117067s FAIL (2025-08-14 22:32:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001127998s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................X.........X.X...............X.......] duration: 6.257979461s FAIL (2025-08-14 22:32:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.258005281s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...................................................XXXXXXXX.] duration: 23.321607548s FAIL (2025-08-14 22:32:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.32162456s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.460026494s FAIL (2025-08-14 22:32:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.460039651s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...X..................X......X................X......X......] duration: 3.261380514s FAIL (2025-08-14 22:32:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 14 22:28:15.058033232 (149h2m9.489113363s) [upload=155017]: Starting upload Aug 14 22:28:15.058069361 (149h2m9.489149482s) [upload=155017]: Done starting upload Aug 14 22:28:15.058119347 (149h2m9.489199478s) [upload=155017]: Uploading segments... Aug 14 22:28:15.058162900 (149h2m9.489243021s) [upload=155017]: Got next segment. Inline: true Aug 14 22:28:15.058166457 (149h2m9.489246588s) [upload=155017]: Waiting for error group managing segments... Aug 14 22:28:15.058197747 (149h2m9.489277878s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:28:15.058201274 (149h2m9.489281405s) [upload=155017]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:28:15.058204019 (149h2m9.48928414s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:28:16.900184965 (149h2m11.331265086s) [upload=155017]: Done uploading segments... Aug 14 22:28:16.900202930 (149h2m11.331283061s) [upload=155017]: Upload finished. err: <nil> Duration:* 3.26140414s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.918784386s FAIL (2025-08-14 22:31:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.918806027s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................X.........X.X...............X......] duration: 6.360733675s FAIL (2025-08-14 22:31:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.360759905s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [....X..................X......X................X......X.....] duration: 2.356968014s FAIL (2025-08-14 22:31:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 14 22:28:15.058033232 (149h2m9.489113363s) [upload=155017]: Starting upload Aug 14 22:28:15.058069361 (149h2m9.489149482s) [upload=155017]: Done starting upload Aug 14 22:28:15.058119347 (149h2m9.489199478s) [upload=155017]: Uploading segments... Aug 14 22:28:15.058162900 (149h2m9.489243021s) [upload=155017]: Got next segment. Inline: true Aug 14 22:28:15.058166457 (149h2m9.489246588s) [upload=155017]: Waiting for error group managing segments... Aug 14 22:28:15.058197747 (149h2m9.489277878s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:28:15.058201274 (149h2m9.489281405s) [upload=155017]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:28:15.058204019 (149h2m9.48928414s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:28:16.900184965 (149h2m11.331265086s) [upload=155017]: Done uploading segments... Aug 14 22:28:16.900202930 (149h2m11.331283061s) [upload=155017]: Upload finished. err: <nil> Duration:* 2.356996838s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 30.001202349s FAIL (2025-08-14 22:31:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001225754s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.....X..................X......X................X......X....] duration: 2.587052034s FAIL (2025-08-14 22:30:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 14 22:28:15.058033232 (149h2m9.489113363s) [upload=155017]: Starting upload Aug 14 22:28:15.058069361 (149h2m9.489149482s) [upload=155017]: Done starting upload Aug 14 22:28:15.058119347 (149h2m9.489199478s) [upload=155017]: Uploading segments... Aug 14 22:28:15.058162900 (149h2m9.489243021s) [upload=155017]: Got next segment. Inline: true Aug 14 22:28:15.058166457 (149h2m9.489246588s) [upload=155017]: Waiting for error group managing segments... Aug 14 22:28:15.058197747 (149h2m9.489277878s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:28:15.058201274 (149h2m9.489281405s) [upload=155017]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:28:15.058204019 (149h2m9.48928414s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:28:16.900184965 (149h2m11.331265086s) [upload=155017]: Done uploading segments... Aug 14 22:28:16.900202930 (149h2m11.331283061s) [upload=155017]: Upload finished. err: <nil> Duration:* 2.587077393s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 14.207516619s FAIL (2025-08-14 22:30:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.207535226s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [......X..................X......X................X......X...] duration: 2.96163722s FAIL (2025-08-14 22:30:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 14 22:28:15.058033232 (149h2m9.489113363s) [upload=155017]: Starting upload Aug 14 22:28:15.058069361 (149h2m9.489149482s) [upload=155017]: Done starting upload Aug 14 22:28:15.058119347 (149h2m9.489199478s) [upload=155017]: Uploading segments... Aug 14 22:28:15.058162900 (149h2m9.489243021s) [upload=155017]: Got next segment. Inline: true Aug 14 22:28:15.058166457 (149h2m9.489246588s) [upload=155017]: Waiting for error group managing segments... Aug 14 22:28:15.058197747 (149h2m9.489277878s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:28:15.058201274 (149h2m9.489281405s) [upload=155017]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:28:15.058204019 (149h2m9.48928414s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:28:16.900184965 (149h2m11.331265086s) [upload=155017]: Done uploading segments... Aug 14 22:28:16.900202930 (149h2m11.331283061s) [upload=155017]: Upload finished. err: <nil> Duration:* 2.96166323s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................X.........X.X...............X.....] duration: 7.415888937s FAIL (2025-08-14 22:30:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 7.415918815s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 30.001106891s FAIL (2025-08-14 22:30:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001131799s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.......X..................X......X................X......X..] duration: 2.754721923s FAIL (2025-08-14 22:29:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 14 22:28:15.058033232 (149h2m9.489113363s) [upload=155017]: Starting upload Aug 14 22:28:15.058069361 (149h2m9.489149482s) [upload=155017]: Done starting upload Aug 14 22:28:15.058119347 (149h2m9.489199478s) [upload=155017]: Uploading segments... Aug 14 22:28:15.058162900 (149h2m9.489243021s) [upload=155017]: Got next segment. Inline: true Aug 14 22:28:15.058166457 (149h2m9.489246588s) [upload=155017]: Waiting for error group managing segments... Aug 14 22:28:15.058197747 (149h2m9.489277878s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:28:15.058201274 (149h2m9.489281405s) [upload=155017]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:28:15.058204019 (149h2m9.48928414s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:28:16.900184965 (149h2m11.331265086s) [upload=155017]: Done uploading segments... Aug 14 22:28:16.900202930 (149h2m11.331283061s) [upload=155017]: Upload finished. err: <nil> Duration:* 2.754747843s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.000358351s FAIL (2025-08-14 22:29:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000376246s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.509479368s FAIL (2025-08-14 22:29:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.509498155s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................X.........X.X...............X....] duration: 6.965253711s FAIL (2025-08-14 22:29:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.965271646s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 30.000939681s FAIL (2025-08-14 22:29:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001224766s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [........X..................X......X................X......X.] duration: 2.303673583s FAIL (2025-08-14 22:29:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 14 22:28:15.058033232 (149h2m9.489113363s) [upload=155017]: Starting upload Aug 14 22:28:15.058069361 (149h2m9.489149482s) [upload=155017]: Done starting upload Aug 14 22:28:15.058119347 (149h2m9.489199478s) [upload=155017]: Uploading segments... Aug 14 22:28:15.058162900 (149h2m9.489243021s) [upload=155017]: Got next segment. Inline: true Aug 14 22:28:15.058166457 (149h2m9.489246588s) [upload=155017]: Waiting for error group managing segments... Aug 14 22:28:15.058197747 (149h2m9.489277878s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:28:15.058201274 (149h2m9.489281405s) [upload=155017]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:28:15.058204019 (149h2m9.48928414s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:28:16.900184965 (149h2m11.331265086s) [upload=155017]: Done uploading segments... Aug 14 22:28:16.900202930 (149h2m11.331283061s) [upload=155017]: Upload finished. err: <nil> Duration:* 2.303698562s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.001143992s FAIL (2025-08-14 22:28:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001161756s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.722281506s FAIL (2025-08-14 22:28:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.722305282s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.........X..................X......X................X......X] duration: 3.684415924s FAIL (2025-08-14 22:28:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 14 22:28:15.058033232 (149h2m9.489113363s) [upload=155017]: Starting upload Aug 14 22:28:15.058069361 (149h2m9.489149482s) [upload=155017]: Done starting upload Aug 14 22:28:15.058119347 (149h2m9.489199478s) [upload=155017]: Uploading segments... Aug 14 22:28:15.058162900 (149h2m9.489243021s) [upload=155017]: Got next segment. Inline: true Aug 14 22:28:15.058166457 (149h2m9.489246588s) [upload=155017]: Waiting for error group managing segments... Aug 14 22:28:15.058197747 (149h2m9.489277878s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:28:15.058201274 (149h2m9.489281405s) [upload=155017]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:28:15.058204019 (149h2m9.48928414s) [upload=155017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:28:16.900184965 (149h2m11.331265086s) [upload=155017]: Done uploading segments... Aug 14 22:28:16.900202930 (149h2m11.331283061s) [upload=155017]: Upload finished. err: <nil> Duration:* 3.684450019s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................X.........X.X...............X...] duration: 6.62804517s FAIL (2025-08-14 22:28:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.628070009s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........X..................X......X................X......] duration: 2.489489111s FAIL (2025-08-14 22:28:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 14 22:23:20.519465643 (148h57m14.950545775s) [upload=154928]: Starting upload Aug 14 22:23:20.519515960 (148h57m14.950596141s) [upload=154928]: Done starting upload Aug 14 22:23:20.519564794 (148h57m14.950644925s) [upload=154928]: Uploading segments... Aug 14 22:23:20.519610100 (148h57m14.950690271s) [upload=154928]: Got next segment. Inline: true Aug 14 22:23:20.519617184 (148h57m14.950697325s) [upload=154928]: Waiting for error group managing segments... Aug 14 22:23:20.519675906 (148h57m14.950756067s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:23:20.519682468 (148h57m14.95076265s) [upload=154928]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:23:20.519686877 (148h57m14.950767008s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:23:20.735192293 (148h57m15.166272414s) [upload=154928]: Done uploading segments... Aug 14 22:23:20.735208533 (148h57m15.166288654s) [upload=154928]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.489510592s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...........X..................X......X................X.....] duration: 4.597935195s FAIL (2025-08-14 22:27:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 14 22:23:20.519465643 (148h57m14.950545775s) [upload=154928]: Starting upload Aug 14 22:23:20.519515960 (148h57m14.950596141s) [upload=154928]: Done starting upload Aug 14 22:23:20.519564794 (148h57m14.950644925s) [upload=154928]: Uploading segments... Aug 14 22:23:20.519610100 (148h57m14.950690271s) [upload=154928]: Got next segment. Inline: true Aug 14 22:23:20.519617184 (148h57m14.950697325s) [upload=154928]: Waiting for error group managing segments... Aug 14 22:23:20.519675906 (148h57m14.950756067s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:23:20.519682468 (148h57m14.95076265s) [upload=154928]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:23:20.519686877 (148h57m14.950767008s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:23:20.735192293 (148h57m15.166272414s) [upload=154928]: Done uploading segments... Aug 14 22:23:20.735208533 (148h57m15.166288654s) [upload=154928]: Upload finished. err: metaclient: Too Many Requests Duration:* 4.597962076s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.001021167s FAIL (2025-08-14 22:27:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001039673s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................X.........X.X...............X..] duration: 6.553787188s FAIL (2025-08-14 22:27:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.553805954s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 9.206676104s FAIL (2025-08-14 22:27:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.206702516s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................X.........X.X...............X.] duration: 5.894985776s FAIL (2025-08-14 22:26:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 5.895039278s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000963427s FAIL (2025-08-14 22:26:33 UTC) orbiter warning: context deadline exceeded Duration: 30.000981131s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [............X..................X......X................X....] duration: 2.472192054s FAIL (2025-08-14 22:26:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 14 22:23:20.519465643 (148h57m14.950545775s) [upload=154928]: Starting upload Aug 14 22:23:20.519515960 (148h57m14.950596141s) [upload=154928]: Done starting upload Aug 14 22:23:20.519564794 (148h57m14.950644925s) [upload=154928]: Uploading segments... Aug 14 22:23:20.519610100 (148h57m14.950690271s) [upload=154928]: Got next segment. Inline: true Aug 14 22:23:20.519617184 (148h57m14.950697325s) [upload=154928]: Waiting for error group managing segments... Aug 14 22:23:20.519675906 (148h57m14.950756067s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:23:20.519682468 (148h57m14.95076265s) [upload=154928]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:23:20.519686877 (148h57m14.950767008s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:23:20.735192293 (148h57m15.166272414s) [upload=154928]: Done uploading segments... Aug 14 22:23:20.735208533 (148h57m15.166288654s) [upload=154928]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.472217203s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................X.........X.X...............X] duration: 8.617825903s FAIL (2025-08-14 22:26:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 8.617852754s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.831751545s FAIL (2025-08-14 22:26:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.831773017s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................X.........X.X...............] duration: 6.601046388s FAIL (2025-08-14 22:25:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 40a1fca759f9ed4dd0330d3cd812d0ef603d36e27ea20a8daf8d445c75796e12 != before 85b5541083f3e205b37b6fbd4d991eee1e2d82d13a442751d5386118275c6a2d ======LOGS====== Aug 14 22:15:53.229351551 (148h49m47.660431692s) [upload=154795]: Starting upload Aug 14 22:15:53.229411636 (148h49m47.660491767s) [upload=154795]: Done starting upload Aug 14 22:15:53.229502910 (148h49m47.660583041s) [upload=154795]: Uploading segments... Aug 14 22:15:53.229644240 (148h49m47.660724372s) [upload=154795]: Got next segment. Inline: false Aug 14 22:15:53.229685219 (148h49m47.66076535s) [upload=154795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:15:53.229689908 (148h49m47.660770039s) [upload=154795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:15:53.714990104 (148h49m48.146070225s) [seg_pos={0 0}, upload=154795]: Begin upload segment... Aug 14 22:15:53.715033727 (148h49m48.146113858s) [seg_pos={0 0}, upload=154795]: Done begin upload segment. Aug 14 22:15:53.715053305 (148h49m48.146133436s) [noise=true , address=64.49.13.137:4493 , piece_id=WT7BGXDIASW53ONK4Q66365J3R4SOFEVS4VPDLEVWMXCHTDHF46A, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715079966 (148h49m48.146160097s) [noise=true , address=77.90.31.11:4583 , piece_id=TGTXMMFJJSSYNDJSLNYMNTQP3TP2M6LZWUJQIE3KHGGJV7DPQCLQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715410057 (148h49m48.146490189s) [noise=true , address=77.90.31.10:4526 , piece_id=KOGFTTIX3P3XIOHO2ER3FPGZ5SUC3AQJZQQAKGDHOVQYPCW6AKVA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715494830 (148h49m48.146574951s) [noise=true , address=104.36.82.21:4492 , piece_id=F4IW2REWNDRANLOVBAZTSX4HEKDKQGGZH4UFHNCLUWWPUHMXK7GA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715690804 (148h49m48.146770935s) [noise=true , address=64.49.13.170:4728 , piece_id=LZ55UINKOZIKWR33HVA4IJLX7P4A2ZGKQNSDRGYSWY7I6K4CN2UQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715969427 (148h49m48.147049548s) [noise=true , address=77.90.31.16:4759 , piece_id=YBFTNV6POD6AUA33SXC2JA5N7YM6QY27K4M2JCAQKEDGY7KJQHXQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716234355 (148h49m48.147314496s) [noise=true , address=104.36.82.37:4662 , piece_id=G2LFOM6WSJXT47O4SJ7PHYKRQAJGNGOU3DPPZWICDJS6N63I3S5Q, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716503329 (148h49m48.147583461s) [noise=true , address=104.36.82.18:4386 , piece_id=OUPDBR2YTY3BKF7TTMBQO3DPKI5CEADZ6DBXLJFB2MQHCLTXDYLA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716728901 (148h49m48.147809032s) [noise=true , address=104.36.82.60:4368 , piece_id=IAPQ2PBUAGNG4FNVS5O6RSZR3EFFLYRQXROIGSILEBCM4SPZGYJA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716960064 (148h49m48.148040195s) [noise=true , address=104.36.82.37:4663 , piece_id=EU5EIC3YI5KYXQ2VIP6TTQIQRIV6BKVQRYTZDBD4HWBKYNA7Q3MA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717180135 (148h49m48.148260266s) [noise=true , address=104.36.82.5:4864 , piece_id=44VLSSVR4SJHH7NFEHNGWRKYWGW7KSP3YCHFRZF77O7FYNCSG6AA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717376681 (148h49m48.148456802s) [noise=true , address=104.36.82.34:4576 , piece_id=WK6723QBKWVC6HM6AELFXOXR5K7LSMYBJHU7M3QJJESQ5BDEVKKA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717570101 (148h49m48.148650232s) [noise=true , address=104.36.82.19:4417 , piece_id=RS2WKSKDVP3KDV43Y3IARTGAUQVNNIXDNC2IAYWIEU27QXOC72SQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717763771 (148h49m48.148843902s) [noise=true , address=104.36.82.58:4270 , piece_id=JSJHYSUVEFMDZLTUJSYDPWH5HKAE7YLHQ4ZMGQK2AHSKGR67CHCA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717976739 (148h49m48.14905686s) [noise=true , address=104.36.82.36:4620 , piece_id=6ZJD3QBQDHBTVDTUCS4VVZ46PU52VD3RHOJ3FG33AF2FKZHOSM5A, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.718178044 (148h49m48.149258175s) [noise=true , address=104.36.82.62:4415 , piece_id=VA754XRFIX3B4LLF37X75BPSQ27BUXLSRXERMNG4OWPVBUFXVFEQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.718360453 (148h49m48.149440584s) [noise=true , address=104.36.82.60:4369 , piece_id=BNPEYJRUQVMHR4DJUM4AX23XTOAHIMPHF46ZTPI66ALQ5S4QGRCQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.718620791 (148h49m48.149700922s) [noise=true , address=104.36.82.42:4818 , piece_id=KWPEW7VBDJI5UORUSKFMTLJH5ZS3U5434DH2PXNB7WVCKXKP6BMQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:54.862013400 (148h49m49.293093531s) [upload=154795]: Next returned nil segment Aug 14 22:15:54.862038238 (148h49m49.293118369s) [upload=154795]: Waiting for error group managing segments... Aug 14 22:15:55.187144180 (148h49m49.618224311s) [noise=true , address=64.49.13.137:4493 , piece_id=WT7BGXDIASW53ONK4Q66365J3R4SOFEVS4VPDLEVWMXCHTDHF46A, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:55.188246019 (148h49m49.61932615s) [noise=true , address=64.49.13.170:4728 , piece_id=LZ55UINKOZIKWR33HVA4IJLX7P4A2ZGKQNSDRGYSWY7I6K4CN2UQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:55.726179112 (148h49m50.157259233s) [noise=true , address=104.36.82.18:4386 , piece_id=OUPDBR2YTY3BKF7TTMBQO3DPKI5CEADZ6DBXLJFB2MQHCLTXDYLA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:55.851420577 (148h49m50.282500708s) [noise=true , address=104.36.82.34:4576 , piece_id=WK6723QBKWVC6HM6AELFXOXR5K7LSMYBJHU7M3QJJESQ5BDEVKKA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.035323223 (148h49m50.466403344s) [noise=true , address=104.36.82.36:4620 , piece_id=6ZJD3QBQDHBTVDTUCS4VVZ46PU52VD3RHOJ3FG33AF2FKZHOSM5A, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.282636930 (148h49m50.713717061s) [noise=true , address=77.90.31.10:4526 , piece_id=KOGFTTIX3P3XIOHO2ER3FPGZ5SUC3AQJZQQAKGDHOVQYPCW6AKVA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.418747181 (148h49m50.849827302s) [noise=true , address=77.90.31.16:4759 , piece_id=YBFTNV6POD6AUA33SXC2JA5N7YM6QY27K4M2JCAQKEDGY7KJQHXQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.581848566 (148h49m51.012928697s) [noise=true , address=77.90.31.11:4583 , piece_id=TGTXMMFJJSSYNDJSLNYMNTQP3TP2M6LZWUJQIE3KHGGJV7DPQCLQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:57.358150731 (148h49m51.789230862s) [noise=true , address=104.36.82.62:4415 , piece_id=VA754XRFIX3B4LLF37X75BPSQ27BUXLSRXERMNG4OWPVBUFXVFEQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:58.420744913 (148h49m52.851825044s) [noise=true , address=104.36.82.21:4492 , piece_id=F4IW2REWNDRANLOVBAZTSX4HEKDKQGGZH4UFHNCLUWWPUHMXK7GA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:59.019199085 (148h49m53.450279216s) [noise=true , address=104.36.82.58:4270 , piece_id=JSJHYSUVEFMDZLTUJSYDPWH5HKAE7YLHQ4ZMGQK2AHSKGR67CHCA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:59.698578279 (148h49m54.12965841s) [noise=true , address=104.36.82.42:4818 , piece_id=KWPEW7VBDJI5UORUSKFMTLJH5ZS3U5434DH2PXNB7WVCKXKP6BMQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:16:00.449819032 (148h49m54.880899163s) [noise=true , address=104.36.82.19:4417 , piece_id=RS2WKSKDVP3KDV43Y3IARTGAUQVNNIXDNC2IAYWIEU27QXOC72SQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:16:02.295457637 (148h49m56.726537768s) [noise=true , address=104.36.82.37:4662 , piece_id=G2LFOM6WSJXT47O4SJ7PHYKRQAJGNGOU3DPPZWICDJS6N63I3S5Q, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:16:02.295484128 (148h49m56.726564249s) [seg_pos={0 0}, upload=154795]: Segment reached optimal threshold of 14 pieces. Aug 14 22:16:02.295593287 (148h49m56.726673418s) [noise=true , address=104.36.82.37:4663 , piece_id=EU5EIC3YI5KYXQ2VIP6TTQIQRIV6BKVQRYTZDBD4HWBKYNA7Q3MA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: expected piece hash; context canceled; EOF Aug 14 22:16:02.295754505 (148h49m56.726834636s) [noise=true , address=104.36.82.5:4864 , piece_id=44VLSSVR4SJHH7NFEHNGWRKYWGW7KSP3YCHFRZF77O7FYNCSG6AA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: write tcp 188.166.225.170:34000->104.36.82.5:4864: use of closed network connection; write tcp 188.166.225.170:34000->104.36.82.5:4864: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34000->104.36.82.5:4864: use of closed network connection Aug 14 22:16:02.295881599 (148h49m56.72696173s) [noise=true , address=104.36.82.60:4369 , piece_id=BNPEYJRUQVMHR4DJUM4AX23XTOAHIMPHF46ZTPI66ALQ5S4QGRCQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nLksGXGvDwuTWfnmkceuwZNDmrF17iNepHPdYkkvZG2QQUwG7): protocol: read tcp 188.166.225.170:49258->104.36.82.60:4369: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:16:02.295896657 (148h49m56.726976788s) [noise=true , address=104.36.82.60:4368 , piece_id=IAPQ2PBUAGNG4FNVS5O6RSZR3EFFLYRQXROIGSILEBCM4SPZGYJA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Q6suyRuFmW1rG5qq8faCFbCkfAPaNfYQaBv6qhL1DNavUffkB): protocol: read tcp 188.166.225.170:50188->104.36.82.60:4368: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:16:02.295911326 (148h49m56.726991457s) [seg_pos={0 0}, upload=154795]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:16:02.295970338 (148h49m56.727050469s) [upload=154795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:16:02.295974476 (148h49m56.727054607s) [upload=154795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:16:02.689245886 (148h49m57.120326017s) [upload=154795]: Done uploading segments... Aug 14 22:16:02.689266686 (148h49m57.120346817s) [upload=154795]: Upload finished. err: <nil> Duration: 6.601070564s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000891661s FAIL (2025-08-14 22:25:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000905738s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000615182s FAIL (2025-08-14 22:25:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000633407s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............X..................X......X................X...] duration: 2.998080487s FAIL (2025-08-14 22:25:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 14 22:23:20.519465643 (148h57m14.950545775s) [upload=154928]: Starting upload Aug 14 22:23:20.519515960 (148h57m14.950596141s) [upload=154928]: Done starting upload Aug 14 22:23:20.519564794 (148h57m14.950644925s) [upload=154928]: Uploading segments... Aug 14 22:23:20.519610100 (148h57m14.950690271s) [upload=154928]: Got next segment. Inline: true Aug 14 22:23:20.519617184 (148h57m14.950697325s) [upload=154928]: Waiting for error group managing segments... Aug 14 22:23:20.519675906 (148h57m14.950756067s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:23:20.519682468 (148h57m14.95076265s) [upload=154928]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:23:20.519686877 (148h57m14.950767008s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:23:20.735192293 (148h57m15.166272414s) [upload=154928]: Done uploading segments... Aug 14 22:23:20.735208533 (148h57m15.166288654s) [upload=154928]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.998101177s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................X.........X.X..............] duration: 6.33133586s FAIL (2025-08-14 22:25:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 40a1fca759f9ed4dd0330d3cd812d0ef603d36e27ea20a8daf8d445c75796e12 != before 85b5541083f3e205b37b6fbd4d991eee1e2d82d13a442751d5386118275c6a2d ======LOGS====== Aug 14 22:15:53.229351551 (148h49m47.660431692s) [upload=154795]: Starting upload Aug 14 22:15:53.229411636 (148h49m47.660491767s) [upload=154795]: Done starting upload Aug 14 22:15:53.229502910 (148h49m47.660583041s) [upload=154795]: Uploading segments... Aug 14 22:15:53.229644240 (148h49m47.660724372s) [upload=154795]: Got next segment. Inline: false Aug 14 22:15:53.229685219 (148h49m47.66076535s) [upload=154795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:15:53.229689908 (148h49m47.660770039s) [upload=154795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:15:53.714990104 (148h49m48.146070225s) [seg_pos={0 0}, upload=154795]: Begin upload segment... Aug 14 22:15:53.715033727 (148h49m48.146113858s) [seg_pos={0 0}, upload=154795]: Done begin upload segment. Aug 14 22:15:53.715053305 (148h49m48.146133436s) [noise=true , address=64.49.13.137:4493 , piece_id=WT7BGXDIASW53ONK4Q66365J3R4SOFEVS4VPDLEVWMXCHTDHF46A, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715079966 (148h49m48.146160097s) [noise=true , address=77.90.31.11:4583 , piece_id=TGTXMMFJJSSYNDJSLNYMNTQP3TP2M6LZWUJQIE3KHGGJV7DPQCLQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715410057 (148h49m48.146490189s) [noise=true , address=77.90.31.10:4526 , piece_id=KOGFTTIX3P3XIOHO2ER3FPGZ5SUC3AQJZQQAKGDHOVQYPCW6AKVA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715494830 (148h49m48.146574951s) [noise=true , address=104.36.82.21:4492 , piece_id=F4IW2REWNDRANLOVBAZTSX4HEKDKQGGZH4UFHNCLUWWPUHMXK7GA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715690804 (148h49m48.146770935s) [noise=true , address=64.49.13.170:4728 , piece_id=LZ55UINKOZIKWR33HVA4IJLX7P4A2ZGKQNSDRGYSWY7I6K4CN2UQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715969427 (148h49m48.147049548s) [noise=true , address=77.90.31.16:4759 , piece_id=YBFTNV6POD6AUA33SXC2JA5N7YM6QY27K4M2JCAQKEDGY7KJQHXQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716234355 (148h49m48.147314496s) [noise=true , address=104.36.82.37:4662 , piece_id=G2LFOM6WSJXT47O4SJ7PHYKRQAJGNGOU3DPPZWICDJS6N63I3S5Q, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716503329 (148h49m48.147583461s) [noise=true , address=104.36.82.18:4386 , piece_id=OUPDBR2YTY3BKF7TTMBQO3DPKI5CEADZ6DBXLJFB2MQHCLTXDYLA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716728901 (148h49m48.147809032s) [noise=true , address=104.36.82.60:4368 , piece_id=IAPQ2PBUAGNG4FNVS5O6RSZR3EFFLYRQXROIGSILEBCM4SPZGYJA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716960064 (148h49m48.148040195s) [noise=true , address=104.36.82.37:4663 , piece_id=EU5EIC3YI5KYXQ2VIP6TTQIQRIV6BKVQRYTZDBD4HWBKYNA7Q3MA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717180135 (148h49m48.148260266s) [noise=true , address=104.36.82.5:4864 , piece_id=44VLSSVR4SJHH7NFEHNGWRKYWGW7KSP3YCHFRZF77O7FYNCSG6AA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717376681 (148h49m48.148456802s) [noise=true , address=104.36.82.34:4576 , piece_id=WK6723QBKWVC6HM6AELFXOXR5K7LSMYBJHU7M3QJJESQ5BDEVKKA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717570101 (148h49m48.148650232s) [noise=true , address=104.36.82.19:4417 , piece_id=RS2WKSKDVP3KDV43Y3IARTGAUQVNNIXDNC2IAYWIEU27QXOC72SQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717763771 (148h49m48.148843902s) [noise=true , address=104.36.82.58:4270 , piece_id=JSJHYSUVEFMDZLTUJSYDPWH5HKAE7YLHQ4ZMGQK2AHSKGR67CHCA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717976739 (148h49m48.14905686s) [noise=true , address=104.36.82.36:4620 , piece_id=6ZJD3QBQDHBTVDTUCS4VVZ46PU52VD3RHOJ3FG33AF2FKZHOSM5A, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.718178044 (148h49m48.149258175s) [noise=true , address=104.36.82.62:4415 , piece_id=VA754XRFIX3B4LLF37X75BPSQ27BUXLSRXERMNG4OWPVBUFXVFEQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.718360453 (148h49m48.149440584s) [noise=true , address=104.36.82.60:4369 , piece_id=BNPEYJRUQVMHR4DJUM4AX23XTOAHIMPHF46ZTPI66ALQ5S4QGRCQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.718620791 (148h49m48.149700922s) [noise=true , address=104.36.82.42:4818 , piece_id=KWPEW7VBDJI5UORUSKFMTLJH5ZS3U5434DH2PXNB7WVCKXKP6BMQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:54.862013400 (148h49m49.293093531s) [upload=154795]: Next returned nil segment Aug 14 22:15:54.862038238 (148h49m49.293118369s) [upload=154795]: Waiting for error group managing segments... Aug 14 22:15:55.187144180 (148h49m49.618224311s) [noise=true , address=64.49.13.137:4493 , piece_id=WT7BGXDIASW53ONK4Q66365J3R4SOFEVS4VPDLEVWMXCHTDHF46A, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:55.188246019 (148h49m49.61932615s) [noise=true , address=64.49.13.170:4728 , piece_id=LZ55UINKOZIKWR33HVA4IJLX7P4A2ZGKQNSDRGYSWY7I6K4CN2UQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:55.726179112 (148h49m50.157259233s) [noise=true , address=104.36.82.18:4386 , piece_id=OUPDBR2YTY3BKF7TTMBQO3DPKI5CEADZ6DBXLJFB2MQHCLTXDYLA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:55.851420577 (148h49m50.282500708s) [noise=true , address=104.36.82.34:4576 , piece_id=WK6723QBKWVC6HM6AELFXOXR5K7LSMYBJHU7M3QJJESQ5BDEVKKA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.035323223 (148h49m50.466403344s) [noise=true , address=104.36.82.36:4620 , piece_id=6ZJD3QBQDHBTVDTUCS4VVZ46PU52VD3RHOJ3FG33AF2FKZHOSM5A, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.282636930 (148h49m50.713717061s) [noise=true , address=77.90.31.10:4526 , piece_id=KOGFTTIX3P3XIOHO2ER3FPGZ5SUC3AQJZQQAKGDHOVQYPCW6AKVA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.418747181 (148h49m50.849827302s) [noise=true , address=77.90.31.16:4759 , piece_id=YBFTNV6POD6AUA33SXC2JA5N7YM6QY27K4M2JCAQKEDGY7KJQHXQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.581848566 (148h49m51.012928697s) [noise=true , address=77.90.31.11:4583 , piece_id=TGTXMMFJJSSYNDJSLNYMNTQP3TP2M6LZWUJQIE3KHGGJV7DPQCLQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:57.358150731 (148h49m51.789230862s) [noise=true , address=104.36.82.62:4415 , piece_id=VA754XRFIX3B4LLF37X75BPSQ27BUXLSRXERMNG4OWPVBUFXVFEQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:58.420744913 (148h49m52.851825044s) [noise=true , address=104.36.82.21:4492 , piece_id=F4IW2REWNDRANLOVBAZTSX4HEKDKQGGZH4UFHNCLUWWPUHMXK7GA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:59.019199085 (148h49m53.450279216s) [noise=true , address=104.36.82.58:4270 , piece_id=JSJHYSUVEFMDZLTUJSYDPWH5HKAE7YLHQ4ZMGQK2AHSKGR67CHCA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:59.698578279 (148h49m54.12965841s) [noise=true , address=104.36.82.42:4818 , piece_id=KWPEW7VBDJI5UORUSKFMTLJH5ZS3U5434DH2PXNB7WVCKXKP6BMQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:16:00.449819032 (148h49m54.880899163s) [noise=true , address=104.36.82.19:4417 , piece_id=RS2WKSKDVP3KDV43Y3IARTGAUQVNNIXDNC2IAYWIEU27QXOC72SQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:16:02.295457637 (148h49m56.726537768s) [noise=true , address=104.36.82.37:4662 , piece_id=G2LFOM6WSJXT47O4SJ7PHYKRQAJGNGOU3DPPZWICDJS6N63I3S5Q, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:16:02.295484128 (148h49m56.726564249s) [seg_pos={0 0}, upload=154795]: Segment reached optimal threshold of 14 pieces. Aug 14 22:16:02.295593287 (148h49m56.726673418s) [noise=true , address=104.36.82.37:4663 , piece_id=EU5EIC3YI5KYXQ2VIP6TTQIQRIV6BKVQRYTZDBD4HWBKYNA7Q3MA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: expected piece hash; context canceled; EOF Aug 14 22:16:02.295754505 (148h49m56.726834636s) [noise=true , address=104.36.82.5:4864 , piece_id=44VLSSVR4SJHH7NFEHNGWRKYWGW7KSP3YCHFRZF77O7FYNCSG6AA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: write tcp 188.166.225.170:34000->104.36.82.5:4864: use of closed network connection; write tcp 188.166.225.170:34000->104.36.82.5:4864: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34000->104.36.82.5:4864: use of closed network connection Aug 14 22:16:02.295881599 (148h49m56.72696173s) [noise=true , address=104.36.82.60:4369 , piece_id=BNPEYJRUQVMHR4DJUM4AX23XTOAHIMPHF46ZTPI66ALQ5S4QGRCQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nLksGXGvDwuTWfnmkceuwZNDmrF17iNepHPdYkkvZG2QQUwG7): protocol: read tcp 188.166.225.170:49258->104.36.82.60:4369: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:16:02.295896657 (148h49m56.726976788s) [noise=true , address=104.36.82.60:4368 , piece_id=IAPQ2PBUAGNG4FNVS5O6RSZR3EFFLYRQXROIGSILEBCM4SPZGYJA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Q6suyRuFmW1rG5qq8faCFbCkfAPaNfYQaBv6qhL1DNavUffkB): protocol: read tcp 188.166.225.170:50188->104.36.82.60:4368: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:16:02.295911326 (148h49m56.726991457s) [seg_pos={0 0}, upload=154795]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:16:02.295970338 (148h49m56.727050469s) [upload=154795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:16:02.295974476 (148h49m56.727054607s) [upload=154795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:16:02.689245886 (148h49m57.120326017s) [upload=154795]: Done uploading segments... Aug 14 22:16:02.689266686 (148h49m57.120346817s) [upload=154795]: Upload finished. err: <nil> Duration: 6.331360847s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [X.............X..................X......X................X..] duration: 2.690225786s FAIL (2025-08-14 22:25:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 14 22:23:20.519465643 (148h57m14.950545775s) [upload=154928]: Starting upload Aug 14 22:23:20.519515960 (148h57m14.950596141s) [upload=154928]: Done starting upload Aug 14 22:23:20.519564794 (148h57m14.950644925s) [upload=154928]: Uploading segments... Aug 14 22:23:20.519610100 (148h57m14.950690271s) [upload=154928]: Got next segment. Inline: true Aug 14 22:23:20.519617184 (148h57m14.950697325s) [upload=154928]: Waiting for error group managing segments... Aug 14 22:23:20.519675906 (148h57m14.950756067s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:23:20.519682468 (148h57m14.95076265s) [upload=154928]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:23:20.519686877 (148h57m14.950767008s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:23:20.735192293 (148h57m15.166272414s) [upload=154928]: Done uploading segments... Aug 14 22:23:20.735208533 (148h57m15.166288654s) [upload=154928]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.690253509s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................X.........X.X.............] duration: 7.541927974s FAIL (2025-08-14 22:24:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 40a1fca759f9ed4dd0330d3cd812d0ef603d36e27ea20a8daf8d445c75796e12 != before 85b5541083f3e205b37b6fbd4d991eee1e2d82d13a442751d5386118275c6a2d ======LOGS====== Aug 14 22:15:53.229351551 (148h49m47.660431692s) [upload=154795]: Starting upload Aug 14 22:15:53.229411636 (148h49m47.660491767s) [upload=154795]: Done starting upload Aug 14 22:15:53.229502910 (148h49m47.660583041s) [upload=154795]: Uploading segments... Aug 14 22:15:53.229644240 (148h49m47.660724372s) [upload=154795]: Got next segment. Inline: false Aug 14 22:15:53.229685219 (148h49m47.66076535s) [upload=154795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:15:53.229689908 (148h49m47.660770039s) [upload=154795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:15:53.714990104 (148h49m48.146070225s) [seg_pos={0 0}, upload=154795]: Begin upload segment... Aug 14 22:15:53.715033727 (148h49m48.146113858s) [seg_pos={0 0}, upload=154795]: Done begin upload segment. Aug 14 22:15:53.715053305 (148h49m48.146133436s) [noise=true , address=64.49.13.137:4493 , piece_id=WT7BGXDIASW53ONK4Q66365J3R4SOFEVS4VPDLEVWMXCHTDHF46A, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715079966 (148h49m48.146160097s) [noise=true , address=77.90.31.11:4583 , piece_id=TGTXMMFJJSSYNDJSLNYMNTQP3TP2M6LZWUJQIE3KHGGJV7DPQCLQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715410057 (148h49m48.146490189s) [noise=true , address=77.90.31.10:4526 , piece_id=KOGFTTIX3P3XIOHO2ER3FPGZ5SUC3AQJZQQAKGDHOVQYPCW6AKVA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715494830 (148h49m48.146574951s) [noise=true , address=104.36.82.21:4492 , piece_id=F4IW2REWNDRANLOVBAZTSX4HEKDKQGGZH4UFHNCLUWWPUHMXK7GA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715690804 (148h49m48.146770935s) [noise=true , address=64.49.13.170:4728 , piece_id=LZ55UINKOZIKWR33HVA4IJLX7P4A2ZGKQNSDRGYSWY7I6K4CN2UQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.715969427 (148h49m48.147049548s) [noise=true , address=77.90.31.16:4759 , piece_id=YBFTNV6POD6AUA33SXC2JA5N7YM6QY27K4M2JCAQKEDGY7KJQHXQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716234355 (148h49m48.147314496s) [noise=true , address=104.36.82.37:4662 , piece_id=G2LFOM6WSJXT47O4SJ7PHYKRQAJGNGOU3DPPZWICDJS6N63I3S5Q, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716503329 (148h49m48.147583461s) [noise=true , address=104.36.82.18:4386 , piece_id=OUPDBR2YTY3BKF7TTMBQO3DPKI5CEADZ6DBXLJFB2MQHCLTXDYLA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716728901 (148h49m48.147809032s) [noise=true , address=104.36.82.60:4368 , piece_id=IAPQ2PBUAGNG4FNVS5O6RSZR3EFFLYRQXROIGSILEBCM4SPZGYJA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.716960064 (148h49m48.148040195s) [noise=true , address=104.36.82.37:4663 , piece_id=EU5EIC3YI5KYXQ2VIP6TTQIQRIV6BKVQRYTZDBD4HWBKYNA7Q3MA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717180135 (148h49m48.148260266s) [noise=true , address=104.36.82.5:4864 , piece_id=44VLSSVR4SJHH7NFEHNGWRKYWGW7KSP3YCHFRZF77O7FYNCSG6AA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717376681 (148h49m48.148456802s) [noise=true , address=104.36.82.34:4576 , piece_id=WK6723QBKWVC6HM6AELFXOXR5K7LSMYBJHU7M3QJJESQ5BDEVKKA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717570101 (148h49m48.148650232s) [noise=true , address=104.36.82.19:4417 , piece_id=RS2WKSKDVP3KDV43Y3IARTGAUQVNNIXDNC2IAYWIEU27QXOC72SQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717763771 (148h49m48.148843902s) [noise=true , address=104.36.82.58:4270 , piece_id=JSJHYSUVEFMDZLTUJSYDPWH5HKAE7YLHQ4ZMGQK2AHSKGR67CHCA, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.717976739 (148h49m48.14905686s) [noise=true , address=104.36.82.36:4620 , piece_id=6ZJD3QBQDHBTVDTUCS4VVZ46PU52VD3RHOJ3FG33AF2FKZHOSM5A, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.718178044 (148h49m48.149258175s) [noise=true , address=104.36.82.62:4415 , piece_id=VA754XRFIX3B4LLF37X75BPSQ27BUXLSRXERMNG4OWPVBUFXVFEQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.718360453 (148h49m48.149440584s) [noise=true , address=104.36.82.60:4369 , piece_id=BNPEYJRUQVMHR4DJUM4AX23XTOAHIMPHF46ZTPI66ALQ5S4QGRCQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:53.718620791 (148h49m48.149700922s) [noise=true , address=104.36.82.42:4818 , piece_id=KWPEW7VBDJI5UORUSKFMTLJH5ZS3U5434DH2PXNB7WVCKXKP6BMQ, seg_pos={0 0}, upload=154795]: Uploading piece... Aug 14 22:15:54.862013400 (148h49m49.293093531s) [upload=154795]: Next returned nil segment Aug 14 22:15:54.862038238 (148h49m49.293118369s) [upload=154795]: Waiting for error group managing segments... Aug 14 22:15:55.187144180 (148h49m49.618224311s) [noise=true , address=64.49.13.137:4493 , piece_id=WT7BGXDIASW53ONK4Q66365J3R4SOFEVS4VPDLEVWMXCHTDHF46A, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:55.188246019 (148h49m49.61932615s) [noise=true , address=64.49.13.170:4728 , piece_id=LZ55UINKOZIKWR33HVA4IJLX7P4A2ZGKQNSDRGYSWY7I6K4CN2UQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:55.726179112 (148h49m50.157259233s) [noise=true , address=104.36.82.18:4386 , piece_id=OUPDBR2YTY3BKF7TTMBQO3DPKI5CEADZ6DBXLJFB2MQHCLTXDYLA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:55.851420577 (148h49m50.282500708s) [noise=true , address=104.36.82.34:4576 , piece_id=WK6723QBKWVC6HM6AELFXOXR5K7LSMYBJHU7M3QJJESQ5BDEVKKA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.035323223 (148h49m50.466403344s) [noise=true , address=104.36.82.36:4620 , piece_id=6ZJD3QBQDHBTVDTUCS4VVZ46PU52VD3RHOJ3FG33AF2FKZHOSM5A, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.282636930 (148h49m50.713717061s) [noise=true , address=77.90.31.10:4526 , piece_id=KOGFTTIX3P3XIOHO2ER3FPGZ5SUC3AQJZQQAKGDHOVQYPCW6AKVA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.418747181 (148h49m50.849827302s) [noise=true , address=77.90.31.16:4759 , piece_id=YBFTNV6POD6AUA33SXC2JA5N7YM6QY27K4M2JCAQKEDGY7KJQHXQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:56.581848566 (148h49m51.012928697s) [noise=true , address=77.90.31.11:4583 , piece_id=TGTXMMFJJSSYNDJSLNYMNTQP3TP2M6LZWUJQIE3KHGGJV7DPQCLQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:57.358150731 (148h49m51.789230862s) [noise=true , address=104.36.82.62:4415 , piece_id=VA754XRFIX3B4LLF37X75BPSQ27BUXLSRXERMNG4OWPVBUFXVFEQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:58.420744913 (148h49m52.851825044s) [noise=true , address=104.36.82.21:4492 , piece_id=F4IW2REWNDRANLOVBAZTSX4HEKDKQGGZH4UFHNCLUWWPUHMXK7GA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:59.019199085 (148h49m53.450279216s) [noise=true , address=104.36.82.58:4270 , piece_id=JSJHYSUVEFMDZLTUJSYDPWH5HKAE7YLHQ4ZMGQK2AHSKGR67CHCA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:15:59.698578279 (148h49m54.12965841s) [noise=true , address=104.36.82.42:4818 , piece_id=KWPEW7VBDJI5UORUSKFMTLJH5ZS3U5434DH2PXNB7WVCKXKP6BMQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:16:00.449819032 (148h49m54.880899163s) [noise=true , address=104.36.82.19:4417 , piece_id=RS2WKSKDVP3KDV43Y3IARTGAUQVNNIXDNC2IAYWIEU27QXOC72SQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:16:02.295457637 (148h49m56.726537768s) [noise=true , address=104.36.82.37:4662 , piece_id=G2LFOM6WSJXT47O4SJ7PHYKRQAJGNGOU3DPPZWICDJS6N63I3S5Q, seg_pos={0 0}, upload=154795]: Done uploading piece. err: <nil> Aug 14 22:16:02.295484128 (148h49m56.726564249s) [seg_pos={0 0}, upload=154795]: Segment reached optimal threshold of 14 pieces. Aug 14 22:16:02.295593287 (148h49m56.726673418s) [noise=true , address=104.36.82.37:4663 , piece_id=EU5EIC3YI5KYXQ2VIP6TTQIQRIV6BKVQRYTZDBD4HWBKYNA7Q3MA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: expected piece hash; context canceled; EOF Aug 14 22:16:02.295754505 (148h49m56.726834636s) [noise=true , address=104.36.82.5:4864 , piece_id=44VLSSVR4SJHH7NFEHNGWRKYWGW7KSP3YCHFRZF77O7FYNCSG6AA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: write tcp 188.166.225.170:34000->104.36.82.5:4864: use of closed network connection; write tcp 188.166.225.170:34000->104.36.82.5:4864: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34000->104.36.82.5:4864: use of closed network connection Aug 14 22:16:02.295881599 (148h49m56.72696173s) [noise=true , address=104.36.82.60:4369 , piece_id=BNPEYJRUQVMHR4DJUM4AX23XTOAHIMPHF46ZTPI66ALQ5S4QGRCQ, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nLksGXGvDwuTWfnmkceuwZNDmrF17iNepHPdYkkvZG2QQUwG7): protocol: read tcp 188.166.225.170:49258->104.36.82.60:4369: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:16:02.295896657 (148h49m56.726976788s) [noise=true , address=104.36.82.60:4368 , piece_id=IAPQ2PBUAGNG4FNVS5O6RSZR3EFFLYRQXROIGSILEBCM4SPZGYJA, seg_pos={0 0}, upload=154795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Q6suyRuFmW1rG5qq8faCFbCkfAPaNfYQaBv6qhL1DNavUffkB): protocol: read tcp 188.166.225.170:50188->104.36.82.60:4368: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:16:02.295911326 (148h49m56.726991457s) [seg_pos={0 0}, upload=154795]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:16:02.295970338 (148h49m56.727050469s) [upload=154795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:16:02.295974476 (148h49m56.727054607s) [upload=154795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:16:02.689245886 (148h49m57.120326017s) [upload=154795]: Done uploading segments... Aug 14 22:16:02.689266686 (148h49m57.120346817s) [upload=154795]: Upload finished. err: <nil> Duration: 7.541953655s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.X.............X..................X......X................X.] duration: 2.325533509s FAIL (2025-08-14 22:24:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 14 22:23:20.519465643 (148h57m14.950545775s) [upload=154928]: Starting upload Aug 14 22:23:20.519515960 (148h57m14.950596141s) [upload=154928]: Done starting upload Aug 14 22:23:20.519564794 (148h57m14.950644925s) [upload=154928]: Uploading segments... Aug 14 22:23:20.519610100 (148h57m14.950690271s) [upload=154928]: Got next segment. Inline: true Aug 14 22:23:20.519617184 (148h57m14.950697325s) [upload=154928]: Waiting for error group managing segments... Aug 14 22:23:20.519675906 (148h57m14.950756067s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:23:20.519682468 (148h57m14.95076265s) [upload=154928]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:23:20.519686877 (148h57m14.950767008s) [upload=154928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:23:20.735192293 (148h57m15.166272414s) [upload=154928]: Done uploading segments... Aug 14 22:23:20.735208533 (148h57m15.166288654s) [upload=154928]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.325561253s core uplink us-select-1 us1-placement-20 metainfo-projects

1
...
15537
15538
15539
15540
15541
...
17921