Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 896007 reports (Page 15538 of 17921)

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [..................................XXXXXXXX.XXXXXXX.X..XXXXXX] duration: 30.000297565s FAIL (2025-08-14 22:45:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000307073s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X.........X.X...............X.......................] duration: 6.0653477s FAIL (2025-08-14 22:44:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.06537279s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [...................................XXXXXXXX.XXXXXXX.X..XXXXX] duration: 30.000382424s FAIL (2025-08-14 22:44:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000401651s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 4.671613849s FAIL (2025-08-14 22:44:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.671642684s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.........X......X................X......X.......X...........] duration: 2.02076255s FAIL (2025-08-14 22:43:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.020796536s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X.........X.X...............X......................] duration: 6.389732822s FAIL (2025-08-14 22:43:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.389776185s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 14.600413571s FAIL (2025-08-14 22:43:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.600435774s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [....................................XXXXXXXX.XXXXXXX.X..XXXX] duration: 30.001073322s FAIL (2025-08-14 22:43:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001099843s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 15.769589146s FAIL (2025-08-14 22:43:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 15.769605327s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........X.........X.X...............X.....................] duration: 6.533189201s FAIL (2025-08-14 22:42:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.533217214s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [..........X......X................X......X.......X..........] duration: 2.353734135s FAIL (2025-08-14 22:42:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.353797065s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.....................................XXXXXXXX.XXXXXXX.X..XXX] duration: 30.000422512s FAIL (2025-08-14 22:42:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.0004579s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 3.217940405s FAIL (2025-08-14 22:42:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.217966046s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........X.........X.X...............X....................] duration: 6.100346183s FAIL (2025-08-14 22:42:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.100409413s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...........X......X................X......X.......X.........] duration: 2.662617699s FAIL (2025-08-14 22:41:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.662640463s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 4.510171647s FAIL (2025-08-14 22:41:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.51028827s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [......................................XXXXXXXX.XXXXXXX.X..XX] duration: 30.001449736s FAIL (2025-08-14 22:41:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001480915s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X.........X.X...............X...................] duration: 8.169196411s FAIL (2025-08-14 22:41:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 8.169234704s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X.........X.X...............X..................] duration: 8.575929076s FAIL (2025-08-14 22:41:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 8.575954394s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [............X......X................X......X.......X........] duration: 2.469126384s FAIL (2025-08-14 22:40:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.469151363s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X.........X.X...............X.................] duration: 6.65444454s FAIL (2025-08-14 22:40:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.654473013s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 4.030785634s FAIL (2025-08-14 22:40:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.030813827s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.......................................XXXXXXXX.XXXXXXX.X..X] duration: 30.001320575s FAIL (2025-08-14 22:40:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.0013431s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X.........X.X...............X................] duration: 6.57241766s FAIL (2025-08-14 22:39:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.572442187s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.080439011s FAIL (2025-08-14 22:39:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.080457928s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.............X......X................X......X.......X.......] duration: 2.036256427s FAIL (2025-08-14 22:39:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.036311422s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X.........X.X...............X...............] duration: 6.224400222s FAIL (2025-08-14 22:39:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.224444307s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [........................................XXXXXXXX.XXXXXXX.X..] duration: 14.435676028s FAIL (2025-08-14 22:39:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.435688271s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 4.15766873s FAIL (2025-08-14 22:38:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.157688689s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [.........................................XXXXXXXX.XXXXXXX.X.] duration: 24.125711459s FAIL (2025-08-14 22:38:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 24.125723531s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X.........X.X...............X..............] duration: 6.268121696s FAIL (2025-08-14 22:38:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.268146763s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [..........................................XXXXXXXX.XXXXXXX.X] duration: 30.000753043s FAIL (2025-08-14 22:38:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000768482s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [..............X......X................X......X.......X......] duration: 2.794535868s FAIL (2025-08-14 22:38:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.794566087s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...............X......X................X......X.......X.....] duration: 2.101532106s FAIL (2025-08-14 22:37:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.101556953s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 7.671504082s FAIL (2025-08-14 22:37:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.671524981s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................X.........X.X...............X.............] duration: 7.97809647s FAIL (2025-08-14 22:37:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 7.978124123s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [................X......X................X......X.......X....] duration: 2.357860351s FAIL (2025-08-14 22:37:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.357878837s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [...........................................XXXXXXXX.XXXXXXX.] duration: 13.143390533s FAIL (2025-08-14 22:37:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 13.143410671s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [............................................XXXXXXXX.XXXXXXX] duration: 30.001324s FAIL (2025-08-14 22:37:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001335442s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.................X......X................X......X.......X...] duration: 2.200084226s FAIL (2025-08-14 22:36:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.200111237s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................X.........X.X...............X............] duration: 6.496012249s FAIL (2025-08-14 22:36:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 6.49604402s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 12.994915144s FAIL (2025-08-14 22:36:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.994940773s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.............................................XXXXXXXX.XXXXXX] duration: 30.001429994s FAIL (2025-08-14 22:36:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001458088s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [..................X......X................X......X.......X..] duration: 2.638020445s FAIL (2025-08-14 22:36:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.638045141s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..............................................XXXXXXXX.XXXXX] duration: 30.001851733s FAIL (2025-08-14 22:35:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00186065s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.430436821s FAIL (2025-08-14 22:35:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.430454153s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................X.........X.X...............X...........] duration: 7.36142627s FAIL (2025-08-14 22:35:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 584456c42d708087c09f0faeed3d1694451ccddbcd31c02942c9bab22782a5ee != before 5c56d2ec81ba59faf7cfbab91a8ddcef2d8a400fc52930931b158cbc222317fa ======LOGS====== Aug 14 22:26:13.497682333 (149h0m7.928762464s) [upload=154981]: Starting upload Aug 14 22:26:13.497731406 (149h0m7.928811537s) [upload=154981]: Done starting upload Aug 14 22:26:13.497858279 (149h0m7.92893842s) [upload=154981]: Uploading segments... Aug 14 22:26:13.498075795 (149h0m7.929155916s) [upload=154981]: Got next segment. Inline: false Aug 14 22:26:13.498128997 (149h0m7.929209118s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:26:13.498133646 (149h0m7.929213777s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 14 22:26:13.992314623 (149h0m8.423394744s) [seg_pos={0 0}, upload=154981]: Begin upload segment... Aug 14 22:26:13.992365701 (149h0m8.423445832s) [seg_pos={0 0}, upload=154981]: Done begin upload segment. Aug 14 22:26:13.992387002 (149h0m8.423467133s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992454140 (149h0m8.423534271s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.992940611 (149h0m8.424020742s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.993072905 (149h0m8.424153036s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994062568 (149h0m8.425142699s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994133022 (149h0m8.425213153s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994388251 (149h0m8.425468383s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994595357 (149h0m8.425675488s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.994800130 (149h0m8.425880261s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995007416 (149h0m8.426087547s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995221876 (149h0m8.426302007s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995422941 (149h0m8.426503073s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995617393 (149h0m8.426697525s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.995813909 (149h0m8.42689404s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996000406 (149h0m8.427080537s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.996185730 (149h0m8.427265871s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.998605130 (149h0m8.429685251s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:13.999270543 (149h0m8.430350664s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Uploading piece... Aug 14 22:26:15.479210578 (149h0m9.910290699s) [upload=154981]: Next returned nil segment Aug 14 22:26:15.479224896 (149h0m9.910305027s) [upload=154981]: Waiting for error group managing segments... Aug 14 22:26:15.884191582 (149h0m10.315271713s) [noise=true , address=79.127.220.101:4062 , piece_id=EB3WGWWNK3WC5L7QBAXINF45X3QG2UBXKC7NQDIMG2MC37W574IQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:15.979809937 (149h0m10.410890068s) [noise=true , address=104.36.82.22:4519 , piece_id=TRBOMMTO2D6H7IZOIZG7CJ4L7WW7JI67GJCZPEH7GZGOUO3JLT4A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.038643598 (149h0m10.469723719s) [noise=true , address=104.36.82.23:4459 , piece_id=R3VTPCYRCJE5BAQJKDG76UBIZW3WQADZIRPPPVLORFHAPPANQ6RQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.046584533 (149h0m10.477664664s) [noise=true , address=79.127.201.215:4343 , piece_id=32PBVWDURI6ZKXXVR5JUTRFOFZO63EWKAW5ZWKMJAKCSZVTMKPMQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.060061459 (149h0m10.49114159s) [noise=true , address=77.90.31.15:4721 , piece_id=ZDMZ772CRI3JJKIURGTAWR3GERAXUT5PPJ367KMFTGFESLEFWN5Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.109095179 (149h0m10.54017531s) [noise=true , address=104.36.82.2:4751 , piece_id=AN26TMRGCIDRWZJLRZ3D2RAPKGFWYG2SCPMVQIDDBGSULPR3CN3Q, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.125159296 (149h0m10.556239427s) [noise=true , address=104.36.82.5:4868 , piece_id=5IQVXHIE6U5OU7HX7AOKCZCNTAZTSY6OXULF2EIKNLNVEOYSPYCA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.127764371 (149h0m10.558844502s) [noise=true , address=104.36.82.58:4274 , piece_id=3O35Z7OA4PHQELTWQMOWZBU4OYOEGU6PTAV3UOJIRGCEZBNFEGEA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.274353653 (149h0m10.705433784s) [noise=true , address=77.90.31.21:4904 , piece_id=25ESPAPI5EX3PEK5DOKSGQJUY67XA7T5ABRA6KWXUEEYB2HRDITQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.432231163 (149h0m10.863311294s) [noise=true , address=77.90.31.13:4631 , piece_id=4V6PL6A2LX4LU4FX3FQKAIN75TMNSBPLAFIKESAKNGRZQWH2FCMA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.479539401 (149h0m10.910619532s) [noise=true , address=77.90.31.16:4749 , piece_id=YH4S54NRP375O66L7SRBMK3QPGXFUVREEBJOG55JHZALMZBVEYWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.517647347 (149h0m10.948727468s) [noise=true , address=77.90.31.12:4606 , piece_id=JAUXFZRHY3XNV3IU3FGMEDT6ILKAHV6Z7P5EPP4NYFXSVP22CAWA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.698196867 (149h0m11.129276988s) [noise=true , address=104.36.82.60:4359 , piece_id=6AN4XA4S54IBZZOISGGKUIIJXCAEDCOX3UDZBMSMSL47LAMX4VUQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701477545 (149h0m11.132557706s) [noise=true , address=104.36.82.45:4937 , piece_id=N5WKBIV2L5LFW3MWMCHTHQEMXFWK3GS4KVGZ6ZCBYZ57WWFZJUPQ, seg_pos={0 0}, upload=154981]: Done uploading piece. err: <nil> Aug 14 22:26:16.701494006 (149h0m11.132574137s) [seg_pos={0 0}, upload=154981]: Segment reached optimal threshold of 14 pieces. Aug 14 22:26:16.701792186 (149h0m11.132872327s) [noise=true , address=104.36.82.47:4527 , piece_id=AKTBF5SBOIOFPC6XWQ6RIBAUBCBBCPNJFNW7YLGV3IW573EGUOZA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57956->104.36.82.47:4527: use of closed network connection Aug 14 22:26:16.701887078 (149h0m11.132967209s) [noise=true , address=104.36.82.3:4780 , piece_id=JXS3SG57OPOQNGKXKFSOOF4XAH2GNE4SDN2OG2WVZYI6PCC5YGYA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM): protocol: read tcp 188.166.225.170:38298->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702037776 (149h0m11.133117957s) [noise=true , address=104.36.82.3:4784 , piece_id=S2G2YJNQB54DYHVD6JWNQ47PTD2BEYUSF4OLEA3IW2JQFKFBWA3A, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1534abg2Jn4zTQzvBcDDZDLq2QZp6kBKY1q6zEbk2PC8mJrnq3): protocol: read tcp 188.166.225.170:43306->104.36.82.3:4784: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702046854 (149h0m11.133126985s) [noise=true , address=104.36.82.20:4452 , piece_id=KAKFHVV6DYHJGVEKEXXNHINQRWO3WWWRVEGV5WRMX5TW6KFCMBNA, seg_pos={0 0}, upload=154981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HX5YAgmkBYJyw6bAY1vz4ov5aBeDsXjU4s7pxpBKAyLbG8yN6): protocol: read tcp 188.166.225.170:52080->104.36.82.20:4452: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 14 22:26:16.702087111 (149h0m11.133167242s) [seg_pos={0 0}, upload=154981]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 14 22:26:16.702169368 (149h0m11.133249499s) [upload=154981]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 14 22:26:16.702173857 (149h0m11.133253988s) [upload=154981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:26:17.128005086 (149h0m11.559085237s) [upload=154981]: Done uploading segments... Aug 14 22:26:17.128028250 (149h0m11.559108401s) [upload=154981]: Upload finished. err: <nil> Duration: 7.361447019s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [X..................X......X................X......X.......X.] duration: 2.429187416s FAIL (2025-08-14 22:35:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 1e316fd097b1870267c819b3250fd40531bcf53e3a840dd8e2b7b715082be916 != before 5f43c01f2131df9e97049d3f6bbb1f3377e2746aa976a893500957879cf9994d ======LOGS====== Aug 14 22:34:27.071178308 (149h8m21.50225844s) [upload=155123]: Starting upload Aug 14 22:34:27.071219086 (149h8m21.502299217s) [upload=155123]: Done starting upload Aug 14 22:34:27.071267589 (149h8m21.50234772s) [upload=155123]: Uploading segments... Aug 14 22:34:27.071299530 (149h8m21.502379651s) [upload=155123]: Got next segment. Inline: true Aug 14 22:34:27.071303288 (149h8m21.502383419s) [upload=155123]: Waiting for error group managing segments... Aug 14 22:34:27.071336451 (149h8m21.502416582s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 14 22:34:27.071340519 (149h8m21.50242065s) [upload=155123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 14 22:34:27.071343695 (149h8m21.502423826s) [upload=155123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 14 22:34:28.085395588 (149h8m22.516475719s) [upload=155123]: Done uploading segments... Aug 14 22:34:28.085442427 (149h8m22.516522558s) [upload=155123]: Upload finished. err: <nil> Duration:* 2.429212835s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...............................................XXXXXXXX.XXXX] duration: 30.001577198s FAIL (2025-08-14 22:35:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001599089s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 8.40664552s FAIL (2025-08-14 22:34:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.406662815s edge gateway us-select-1

1
...
15536
15537
15538
15539
15540
...
17921