Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.......X.........................X...............X...X......] duration:
6.022953451s
FAIL (2025-07-09 22:55:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.02297912s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...X.....................X..................................] duration:
8.04840009s
FAIL (2025-07-09 22:55:16 UTC)
orbiter warning: context deadline exceeded
Duration: 8.048422744s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....X.....................X.................................] duration:
16.368082667s
FAIL (2025-07-09 22:55:04 UTC)
orbiter warning: context deadline exceeded
Duration: 16.368143372s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [........X.........................X...............X...X.....] duration:
5.906655795s
FAIL (2025-07-09 22:54:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.906681154s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.........X.........X...............................X..X.....] duration:
18.490489784s
FAIL (2025-07-09 22:54:39 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
8]: Done uploading segments...
Jul 9 22:49:03.940425795 (397h38m38.71106713s) [upload=318138]: Upload finished. err: <nil>
Jul 9 22:49:26.942685461 (397h39m1.713326806s) [upload=318145]: Starting upload
Jul 9 22:49:26.942766596 (397h39m1.713407971s) [upload=318145]: Done starting upload
Jul 9 22:49:26.942899851 (397h39m1.713541186s) [upload=318145]: Uploading segments...
Jul 9 22:49:26.943078553 (397h39m1.713719888s) [upload=318145]: Got next segment. Inline: false
Jul 9 22:49:26.943128258 (397h39m1.713769583s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:49:26.943133378 (397h39m1.713774703s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:49:27.391221381 (397h39m2.161862716s) [seg_pos={0 0}, upload=318145]: Begin upload segment...
Jul 9 22:49:27.391272879 (397h39m2.161914215s) [seg_pos={0 0}, upload=318145]: Done begin upload segment.
Jul 9 22:49:27.391293639 (397h39m2.161934974s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391339827 (397h39m2.161981152s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391636765 (397h39m2.16227809s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391776052 (397h39m2.162417387s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391900591 (397h39m2.162541926s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392075054 (397h39m2.162716389s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392182890 (397h39m2.162824216s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392427849 (397h39m2.163069184s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392508023 (397h39m2.163149358s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392699018 (397h39m2.163340343s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392740437 (397h39m2.163381792s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392950770 (397h39m2.163592105s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393004523 (397h39m2.163645858s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393258058 (397h39m2.163899393s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393469623 (397h39m2.164110958s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393671368 (397h39m2.164312703s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393919603 (397h39m2.164560928s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.394128562 (397h39m2.164769897s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:28.541659683 (397h39m3.312301018s) [upload=318145]: Next returned nil segment
Jul 9 22:49:28.541678779 (397h39m3.312320114s) [upload=318145]: Waiting for error group managing segments...
Jul 9 22:49:29.222239328 (397h39m3.992880663s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396477165 (397h39m4.1671185s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396597805 (397h39m4.16723914s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.397196882 (397h39m4.167838227s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.398284352 (397h39m4.168925687s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.407089638 (397h39m4.177730973s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.414636957 (397h39m4.185278292s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:35.309098140 (397h39m10.079739475s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:41.926391228 (397h39m16.697032563s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:46.575760060 (397h39m21.346401385s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:02.779785025 (397h39m37.55042636s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:05.100996812 (397h39m39.871638147s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:20.559293926 (397h39m55.329935261s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:26.299711393 (397h40m1.070352728s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12Mwd9RRYqm3GpbmT4B2KnAd2bXKq7HopJCk3E3W4A79Hh2eypL, address:45.91.95.2:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299775055 (397h40m1.07041639s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1rNDTtTWVFcGgWM4ZWJ4GKhoATGLJTeSUYnBDQKjeRasQ3U8cx, address:45.91.95.5:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299782971 (397h40m1.070424356s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299809722 (397h40m1.070451057s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12rWJM3vC8XrUAfVaoqVZm1qzqcG8AqxLzncExPHEak5oo3QuPA, address:45.91.95.4:4030): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299880867 (397h40m1.070522283s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12USiZrzoPtP51hXzKeyNLg58wgmPCTAKMTvPb5du9CRKFu4FiJ, address:45.91.95.3:4007): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299921666 (397h40m1.070563001s) [seg_pos={0 0}, upload=318145]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 22:50:26.533201557 (397h40m1.303842892s) [upload=318145]: Done uploading segments...
Jul 9 22:50:26.533239139 (397h40m1.303880474s) [upload=318145]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 18.490517287s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....X.....................X................................] duration:
10.272061311s
FAIL (2025-07-09 22:54:18 UTC)
orbiter warning: context deadline exceeded
Duration: 10.272080416s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......X.....................X...............................] duration:
39.492624089s
FAIL (2025-07-09 22:54:08 UTC)
orbiter warning: context deadline exceeded
Duration: 39.492638297s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..........X.........X...............................X..X....] duration:
46.654681713s
FAIL (2025-07-09 22:54:07 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
8]: Done uploading segments...
Jul 9 22:49:03.940425795 (397h38m38.71106713s) [upload=318138]: Upload finished. err: <nil>
Jul 9 22:49:26.942685461 (397h39m1.713326806s) [upload=318145]: Starting upload
Jul 9 22:49:26.942766596 (397h39m1.713407971s) [upload=318145]: Done starting upload
Jul 9 22:49:26.942899851 (397h39m1.713541186s) [upload=318145]: Uploading segments...
Jul 9 22:49:26.943078553 (397h39m1.713719888s) [upload=318145]: Got next segment. Inline: false
Jul 9 22:49:26.943128258 (397h39m1.713769583s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:49:26.943133378 (397h39m1.713774703s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:49:27.391221381 (397h39m2.161862716s) [seg_pos={0 0}, upload=318145]: Begin upload segment...
Jul 9 22:49:27.391272879 (397h39m2.161914215s) [seg_pos={0 0}, upload=318145]: Done begin upload segment.
Jul 9 22:49:27.391293639 (397h39m2.161934974s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391339827 (397h39m2.161981152s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391636765 (397h39m2.16227809s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391776052 (397h39m2.162417387s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391900591 (397h39m2.162541926s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392075054 (397h39m2.162716389s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392182890 (397h39m2.162824216s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392427849 (397h39m2.163069184s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392508023 (397h39m2.163149358s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392699018 (397h39m2.163340343s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392740437 (397h39m2.163381792s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392950770 (397h39m2.163592105s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393004523 (397h39m2.163645858s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393258058 (397h39m2.163899393s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393469623 (397h39m2.164110958s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393671368 (397h39m2.164312703s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393919603 (397h39m2.164560928s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.394128562 (397h39m2.164769897s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:28.541659683 (397h39m3.312301018s) [upload=318145]: Next returned nil segment
Jul 9 22:49:28.541678779 (397h39m3.312320114s) [upload=318145]: Waiting for error group managing segments...
Jul 9 22:49:29.222239328 (397h39m3.992880663s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396477165 (397h39m4.1671185s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396597805 (397h39m4.16723914s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.397196882 (397h39m4.167838227s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.398284352 (397h39m4.168925687s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.407089638 (397h39m4.177730973s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.414636957 (397h39m4.185278292s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:35.309098140 (397h39m10.079739475s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:41.926391228 (397h39m16.697032563s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:46.575760060 (397h39m21.346401385s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:02.779785025 (397h39m37.55042636s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:05.100996812 (397h39m39.871638147s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:20.559293926 (397h39m55.329935261s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:26.299711393 (397h40m1.070352728s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12Mwd9RRYqm3GpbmT4B2KnAd2bXKq7HopJCk3E3W4A79Hh2eypL, address:45.91.95.2:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299775055 (397h40m1.07041639s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1rNDTtTWVFcGgWM4ZWJ4GKhoATGLJTeSUYnBDQKjeRasQ3U8cx, address:45.91.95.5:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299782971 (397h40m1.070424356s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299809722 (397h40m1.070451057s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12rWJM3vC8XrUAfVaoqVZm1qzqcG8AqxLzncExPHEak5oo3QuPA, address:45.91.95.4:4030): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299880867 (397h40m1.070522283s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12USiZrzoPtP51hXzKeyNLg58wgmPCTAKMTvPb5du9CRKFu4FiJ, address:45.91.95.3:4007): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299921666 (397h40m1.070563001s) [seg_pos={0 0}, upload=318145]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 22:50:26.533201557 (397h40m1.303842892s) [upload=318145]: Done uploading segments...
Jul 9 22:50:26.533239139 (397h40m1.303880474s) [upload=318145]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 46.654701953s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.........X.........................X...............X...X....] duration:
6.134284751s
FAIL (2025-07-09 22:53:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.134305169s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......X.....................X..............................] duration:
30.397723594s
FAIL (2025-07-09 22:53:28 UTC)
orbiter warning: context deadline exceeded
Duration: 30.397754463s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...........X.........X...............................X..X...] duration:
30.021379063s
FAIL (2025-07-09 22:53:20 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
8]: Done uploading segments...
Jul 9 22:49:03.940425795 (397h38m38.71106713s) [upload=318138]: Upload finished. err: <nil>
Jul 9 22:49:26.942685461 (397h39m1.713326806s) [upload=318145]: Starting upload
Jul 9 22:49:26.942766596 (397h39m1.713407971s) [upload=318145]: Done starting upload
Jul 9 22:49:26.942899851 (397h39m1.713541186s) [upload=318145]: Uploading segments...
Jul 9 22:49:26.943078553 (397h39m1.713719888s) [upload=318145]: Got next segment. Inline: false
Jul 9 22:49:26.943128258 (397h39m1.713769583s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:49:26.943133378 (397h39m1.713774703s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:49:27.391221381 (397h39m2.161862716s) [seg_pos={0 0}, upload=318145]: Begin upload segment...
Jul 9 22:49:27.391272879 (397h39m2.161914215s) [seg_pos={0 0}, upload=318145]: Done begin upload segment.
Jul 9 22:49:27.391293639 (397h39m2.161934974s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391339827 (397h39m2.161981152s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391636765 (397h39m2.16227809s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391776052 (397h39m2.162417387s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391900591 (397h39m2.162541926s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392075054 (397h39m2.162716389s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392182890 (397h39m2.162824216s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392427849 (397h39m2.163069184s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392508023 (397h39m2.163149358s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392699018 (397h39m2.163340343s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392740437 (397h39m2.163381792s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392950770 (397h39m2.163592105s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393004523 (397h39m2.163645858s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393258058 (397h39m2.163899393s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393469623 (397h39m2.164110958s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393671368 (397h39m2.164312703s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393919603 (397h39m2.164560928s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.394128562 (397h39m2.164769897s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:28.541659683 (397h39m3.312301018s) [upload=318145]: Next returned nil segment
Jul 9 22:49:28.541678779 (397h39m3.312320114s) [upload=318145]: Waiting for error group managing segments...
Jul 9 22:49:29.222239328 (397h39m3.992880663s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396477165 (397h39m4.1671185s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396597805 (397h39m4.16723914s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.397196882 (397h39m4.167838227s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.398284352 (397h39m4.168925687s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.407089638 (397h39m4.177730973s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.414636957 (397h39m4.185278292s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:35.309098140 (397h39m10.079739475s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:41.926391228 (397h39m16.697032563s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:46.575760060 (397h39m21.346401385s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:02.779785025 (397h39m37.55042636s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:05.100996812 (397h39m39.871638147s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:20.559293926 (397h39m55.329935261s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:26.299711393 (397h40m1.070352728s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12Mwd9RRYqm3GpbmT4B2KnAd2bXKq7HopJCk3E3W4A79Hh2eypL, address:45.91.95.2:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299775055 (397h40m1.07041639s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1rNDTtTWVFcGgWM4ZWJ4GKhoATGLJTeSUYnBDQKjeRasQ3U8cx, address:45.91.95.5:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299782971 (397h40m1.070424356s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299809722 (397h40m1.070451057s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12rWJM3vC8XrUAfVaoqVZm1qzqcG8AqxLzncExPHEak5oo3QuPA, address:45.91.95.4:4030): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299880867 (397h40m1.070522283s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12USiZrzoPtP51hXzKeyNLg58wgmPCTAKMTvPb5du9CRKFu4FiJ, address:45.91.95.3:4007): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299921666 (397h40m1.070563001s) [seg_pos={0 0}, upload=318145]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 22:50:26.533201557 (397h40m1.303842892s) [upload=318145]: Done uploading segments...
Jul 9 22:50:26.533239139 (397h40m1.303880474s) [upload=318145]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 30.021404981s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [..........X.........................X...............X...X...] duration:
5.875526555s
FAIL (2025-07-09 22:53:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.875540682s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........X.....................X.............................] duration:
21.635085179s
FAIL (2025-07-09 22:52:56 UTC)
orbiter warning: context deadline exceeded
Duration: 21.635107112s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [...........X.........................X...............X...X..] duration:
5.998938449s
FAIL (2025-07-09 22:52:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.998957975s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.........X.....................X............................] duration:
11.547550031s
FAIL (2025-07-09 22:52:28 UTC)
orbiter warning: context deadline exceeded
Duration: 11.547569468s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [............X.........................X...............X...X.] duration:
5.921600271s
FAIL (2025-07-09 22:52:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.921620379s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [............X.........X...............................X..X..] duration:
19.223637152s
FAIL (2025-07-09 22:51:55 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
8]: Done uploading segments...
Jul 9 22:49:03.940425795 (397h38m38.71106713s) [upload=318138]: Upload finished. err: <nil>
Jul 9 22:49:26.942685461 (397h39m1.713326806s) [upload=318145]: Starting upload
Jul 9 22:49:26.942766596 (397h39m1.713407971s) [upload=318145]: Done starting upload
Jul 9 22:49:26.942899851 (397h39m1.713541186s) [upload=318145]: Uploading segments...
Jul 9 22:49:26.943078553 (397h39m1.713719888s) [upload=318145]: Got next segment. Inline: false
Jul 9 22:49:26.943128258 (397h39m1.713769583s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:49:26.943133378 (397h39m1.713774703s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:49:27.391221381 (397h39m2.161862716s) [seg_pos={0 0}, upload=318145]: Begin upload segment...
Jul 9 22:49:27.391272879 (397h39m2.161914215s) [seg_pos={0 0}, upload=318145]: Done begin upload segment.
Jul 9 22:49:27.391293639 (397h39m2.161934974s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391339827 (397h39m2.161981152s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391636765 (397h39m2.16227809s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391776052 (397h39m2.162417387s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391900591 (397h39m2.162541926s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392075054 (397h39m2.162716389s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392182890 (397h39m2.162824216s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392427849 (397h39m2.163069184s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392508023 (397h39m2.163149358s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392699018 (397h39m2.163340343s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392740437 (397h39m2.163381792s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392950770 (397h39m2.163592105s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393004523 (397h39m2.163645858s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393258058 (397h39m2.163899393s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393469623 (397h39m2.164110958s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393671368 (397h39m2.164312703s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393919603 (397h39m2.164560928s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.394128562 (397h39m2.164769897s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:28.541659683 (397h39m3.312301018s) [upload=318145]: Next returned nil segment
Jul 9 22:49:28.541678779 (397h39m3.312320114s) [upload=318145]: Waiting for error group managing segments...
Jul 9 22:49:29.222239328 (397h39m3.992880663s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396477165 (397h39m4.1671185s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396597805 (397h39m4.16723914s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.397196882 (397h39m4.167838227s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.398284352 (397h39m4.168925687s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.407089638 (397h39m4.177730973s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.414636957 (397h39m4.185278292s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:35.309098140 (397h39m10.079739475s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:41.926391228 (397h39m16.697032563s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:46.575760060 (397h39m21.346401385s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:02.779785025 (397h39m37.55042636s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:05.100996812 (397h39m39.871638147s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:20.559293926 (397h39m55.329935261s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:26.299711393 (397h40m1.070352728s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12Mwd9RRYqm3GpbmT4B2KnAd2bXKq7HopJCk3E3W4A79Hh2eypL, address:45.91.95.2:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299775055 (397h40m1.07041639s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1rNDTtTWVFcGgWM4ZWJ4GKhoATGLJTeSUYnBDQKjeRasQ3U8cx, address:45.91.95.5:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299782971 (397h40m1.070424356s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299809722 (397h40m1.070451057s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12rWJM3vC8XrUAfVaoqVZm1qzqcG8AqxLzncExPHEak5oo3QuPA, address:45.91.95.4:4030): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299880867 (397h40m1.070522283s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12USiZrzoPtP51hXzKeyNLg58wgmPCTAKMTvPb5du9CRKFu4FiJ, address:45.91.95.3:4007): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299921666 (397h40m1.070563001s) [seg_pos={0 0}, upload=318145]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 22:50:26.533201557 (397h40m1.303842892s) [upload=318145]: Done uploading segments...
Jul 9 22:50:26.533239139 (397h40m1.303880474s) [upload=318145]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 19.223661098s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.............X.........................X...............X...X] duration:
30.000760312s
FAIL (2025-07-09 22:51:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000780871s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
4.996638262s
FAIL (2025-07-09 22:51:26 UTC)
orbiter warning: uplink: encryption: context deadline exceeded
Duration: 4.996665073s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..........X.....................X...........................] duration:
13.265176562s
FAIL (2025-07-09 22:51:24 UTC)
orbiter warning: context deadline exceeded
Duration: 13.265214976s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [X.............X.........................X...............X...] duration:
6.455341478s
FAIL (2025-07-09 22:50:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.455360193s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...........X.....................X..........................] duration:
26.455712824s
FAIL (2025-07-09 22:50:48 UTC)
orbiter warning: context deadline exceeded
Duration: 26.455762349s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [X............X.........X...............................X..X.] duration:
17.063814036s
FAIL (2025-07-09 22:50:43 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
8]: Done uploading segments...
Jul 9 22:49:03.940425795 (397h38m38.71106713s) [upload=318138]: Upload finished. err: <nil>
Jul 9 22:49:26.942685461 (397h39m1.713326806s) [upload=318145]: Starting upload
Jul 9 22:49:26.942766596 (397h39m1.713407971s) [upload=318145]: Done starting upload
Jul 9 22:49:26.942899851 (397h39m1.713541186s) [upload=318145]: Uploading segments...
Jul 9 22:49:26.943078553 (397h39m1.713719888s) [upload=318145]: Got next segment. Inline: false
Jul 9 22:49:26.943128258 (397h39m1.713769583s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:49:26.943133378 (397h39m1.713774703s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:49:27.391221381 (397h39m2.161862716s) [seg_pos={0 0}, upload=318145]: Begin upload segment...
Jul 9 22:49:27.391272879 (397h39m2.161914215s) [seg_pos={0 0}, upload=318145]: Done begin upload segment.
Jul 9 22:49:27.391293639 (397h39m2.161934974s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391339827 (397h39m2.161981152s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391636765 (397h39m2.16227809s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391776052 (397h39m2.162417387s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391900591 (397h39m2.162541926s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392075054 (397h39m2.162716389s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392182890 (397h39m2.162824216s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392427849 (397h39m2.163069184s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392508023 (397h39m2.163149358s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392699018 (397h39m2.163340343s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392740437 (397h39m2.163381792s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392950770 (397h39m2.163592105s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393004523 (397h39m2.163645858s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393258058 (397h39m2.163899393s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393469623 (397h39m2.164110958s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393671368 (397h39m2.164312703s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393919603 (397h39m2.164560928s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.394128562 (397h39m2.164769897s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:28.541659683 (397h39m3.312301018s) [upload=318145]: Next returned nil segment
Jul 9 22:49:28.541678779 (397h39m3.312320114s) [upload=318145]: Waiting for error group managing segments...
Jul 9 22:49:29.222239328 (397h39m3.992880663s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396477165 (397h39m4.1671185s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396597805 (397h39m4.16723914s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.397196882 (397h39m4.167838227s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.398284352 (397h39m4.168925687s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.407089638 (397h39m4.177730973s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.414636957 (397h39m4.185278292s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:35.309098140 (397h39m10.079739475s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:41.926391228 (397h39m16.697032563s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:46.575760060 (397h39m21.346401385s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:02.779785025 (397h39m37.55042636s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:05.100996812 (397h39m39.871638147s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:20.559293926 (397h39m55.329935261s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:26.299711393 (397h40m1.070352728s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12Mwd9RRYqm3GpbmT4B2KnAd2bXKq7HopJCk3E3W4A79Hh2eypL, address:45.91.95.2:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299775055 (397h40m1.07041639s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1rNDTtTWVFcGgWM4ZWJ4GKhoATGLJTeSUYnBDQKjeRasQ3U8cx, address:45.91.95.5:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299782971 (397h40m1.070424356s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299809722 (397h40m1.070451057s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12rWJM3vC8XrUAfVaoqVZm1qzqcG8AqxLzncExPHEak5oo3QuPA, address:45.91.95.4:4030): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299880867 (397h40m1.070522283s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12USiZrzoPtP51hXzKeyNLg58wgmPCTAKMTvPb5du9CRKFu4FiJ, address:45.91.95.3:4007): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299921666 (397h40m1.070563001s) [seg_pos={0 0}, upload=318145]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 22:50:26.533201557 (397h40m1.303842892s) [upload=318145]: Done uploading segments...
Jul 9 22:50:26.533239139 (397h40m1.303880474s) [upload=318145]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 17.063827522s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.X............X.........X...............................X..X] duration:
1m0.235116941s
FAIL (2025-07-09 22:50:26 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
8]: Done uploading segments...
Jul 9 22:49:03.940425795 (397h38m38.71106713s) [upload=318138]: Upload finished. err: <nil>
Jul 9 22:49:26.942685461 (397h39m1.713326806s) [upload=318145]: Starting upload
Jul 9 22:49:26.942766596 (397h39m1.713407971s) [upload=318145]: Done starting upload
Jul 9 22:49:26.942899851 (397h39m1.713541186s) [upload=318145]: Uploading segments...
Jul 9 22:49:26.943078553 (397h39m1.713719888s) [upload=318145]: Got next segment. Inline: false
Jul 9 22:49:26.943128258 (397h39m1.713769583s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:49:26.943133378 (397h39m1.713774703s) [upload=318145]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:49:27.391221381 (397h39m2.161862716s) [seg_pos={0 0}, upload=318145]: Begin upload segment...
Jul 9 22:49:27.391272879 (397h39m2.161914215s) [seg_pos={0 0}, upload=318145]: Done begin upload segment.
Jul 9 22:49:27.391293639 (397h39m2.161934974s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391339827 (397h39m2.161981152s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391636765 (397h39m2.16227809s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391776052 (397h39m2.162417387s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.391900591 (397h39m2.162541926s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392075054 (397h39m2.162716389s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392182890 (397h39m2.162824216s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392427849 (397h39m2.163069184s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392508023 (397h39m2.163149358s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392699018 (397h39m2.163340343s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392740437 (397h39m2.163381792s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.392950770 (397h39m2.163592105s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393004523 (397h39m2.163645858s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393258058 (397h39m2.163899393s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393469623 (397h39m2.164110958s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393671368 (397h39m2.164312703s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.393919603 (397h39m2.164560928s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:27.394128562 (397h39m2.164769897s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Uploading piece...
Jul 9 22:49:28.541659683 (397h39m3.312301018s) [upload=318145]: Next returned nil segment
Jul 9 22:49:28.541678779 (397h39m3.312320114s) [upload=318145]: Waiting for error group managing segments...
Jul 9 22:49:29.222239328 (397h39m3.992880663s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396477165 (397h39m4.1671185s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.396597805 (397h39m4.16723914s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.397196882 (397h39m4.167838227s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.398284352 (397h39m4.168925687s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.407089638 (397h39m4.177730973s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:29.414636957 (397h39m4.185278292s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:35.309098140 (397h39m10.079739475s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:41.926391228 (397h39m16.697032563s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:49:46.575760060 (397h39m21.346401385s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:02.779785025 (397h39m37.55042636s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:05.100996812 (397h39m39.871638147s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:20.559293926 (397h39m55.329935261s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil>
Jul 9 22:50:26.299711393 (397h40m1.070352728s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12Mwd9RRYqm3GpbmT4B2KnAd2bXKq7HopJCk3E3W4A79Hh2eypL, address:45.91.95.2:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299775055 (397h40m1.07041639s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1rNDTtTWVFcGgWM4ZWJ4GKhoATGLJTeSUYnBDQKjeRasQ3U8cx, address:45.91.95.5:4031): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299782971 (397h40m1.070424356s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299809722 (397h40m1.070451057s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12rWJM3vC8XrUAfVaoqVZm1qzqcG8AqxLzncExPHEak5oo3QuPA, address:45.91.95.4:4030): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299880867 (397h40m1.070522283s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12USiZrzoPtP51hXzKeyNLg58wgmPCTAKMTvPb5du9CRKFu4FiJ, address:45.91.95.3:4007): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:50:26.299921666 (397h40m1.070563001s) [seg_pos={0 0}, upload=318145]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 22:50:26.533201557 (397h40m1.303842892s) [upload=318145]: Done uploading segments...
Jul 9 22:50:26.533239139 (397h40m1.303880474s) [upload=318145]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 1m0.235137412s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [............X.....................X.........................] duration:
29.106188277s
FAIL (2025-07-09 22:50:21 UTC)
orbiter warning: context deadline exceeded
Duration: 29.106201723s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
4.820720828s
FAIL (2025-07-09 22:50:17 UTC)
orbiter warning: uplink: encryption: context deadline exceeded
Duration: 4.820752388s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.X.............X.........................X...............X..] duration:
6.129208189s
FAIL (2025-07-09 22:50:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.129223127s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.............X.....................X........................] duration:
46.796567108s
FAIL (2025-07-09 22:49:52 UTC)
orbiter warning: context deadline exceeded
Duration: 46.796588209s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
4.667116136s
FAIL (2025-07-09 22:49:19 UTC)
orbiter warning: uplink: encryption: context deadline exceeded
Duration: 4.667142668s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [X.X.............X.........................X...............X.] duration:
5.953076081s
FAIL (2025-07-09 22:49:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.953097002s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [X.X............X.........X...............................X..] duration:
13.585434975s
FAIL (2025-07-09 22:49:06 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
:03.248453150 (397h36m38.019094495s) [upload=318103]: Upload finished. err: <nil>
Jul 9 22:47:07.946019126 (397h36m42.716660461s) [upload=318117]: Starting upload
Jul 9 22:47:07.946077277 (397h36m42.716718612s) [upload=318117]: Done starting upload
Jul 9 22:47:07.946189321 (397h36m42.716830656s) [upload=318117]: Uploading segments...
Jul 9 22:47:07.946280796 (397h36m42.716922131s) [upload=318117]: Got next segment. Inline: false
Jul 9 22:47:07.946333867 (397h36m42.716975192s) [upload=318117]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:47:07.946338967 (397h36m42.716980302s) [upload=318117]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:47:08.388438504 (397h36m43.159079839s) [seg_pos={0 0}, upload=318117]: Begin upload segment...
Jul 9 22:47:08.388478570 (397h36m43.159119905s) [seg_pos={0 0}, upload=318117]: Done begin upload segment.
Jul 9 22:47:08.388502175 (397h36m43.15914351s) [noise=true , address=79.127.205.233:4629 , piece_id=SCMFOAOYWF3I6ICMJ75QAQROZJIQYISZBU6NHKV7NQ75YNRWNDWQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.388508116 (397h36m43.159149451s) [noise=true , address=104.36.82.36:4638 , piece_id=I4I6P7RDI2OC3Z5FJP6ZUWPSF3UNCMHVRZHEQPDRAI7YDREWM3OA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.388551971 (397h36m43.159193296s) [noise=true , address=45.91.95.3:4029 , piece_id=KIDHLHXLDN6ZFTABE6PFO26IU4SRF4XX3HUG556LDKA2IN5LIYMQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.388844890 (397h36m43.159486215s) [noise=true , address=104.36.82.20:4453 , piece_id=53UPZW3K64NMBD2VWD3WFIDQUPL2D2HB67CUHMCXFAVTBCPPF3YQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389067626 (397h36m43.159708951s) [noise=true , address=77.90.31.17:4800 , piece_id=65FVQXPTFSDSBM7OF2HZLVPIQRUH3L4IA3VQTPQCSGYURTFP3CWA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389087926 (397h36m43.159729251s) [noise=true , address=104.36.82.60:4363 , piece_id=R3L3AUCM5G4NDXTEEVMA3LUMAPVO2RK6YPYCPLOVGLFHSOVE35YA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389330299 (397h36m43.159971624s) [noise=true , address=104.36.82.38:4724 , piece_id=ZQVGYE67IKOQFNIIGKQA3VRGKIODUD6AQIM6CPMAE2MZEDPMRLJQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389524060 (397h36m43.160165395s) [noise=true , address=45.91.95.2:4002 , piece_id=V6WN7U6WI67BNTETWNJGST5TDPLR6BZHI5JM6MEVD27HVW6O3ERA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389567773 (397h36m43.160209098s) [noise=true , address=77.90.31.11:4580 , piece_id=TY4475YRV6Z7BJ5GUQPNAVIPL3IDSTUGNSYQN3COP3YF7AAINKLA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389825587 (397h36m43.160466922s) [noise=true , address=104.36.82.19:4414 , piece_id=R4MZB3S2TVIGFQAPOOZVTRUWEAITFVMJ2BOD2ENGFISBZ6SQOXZQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389985021 (397h36m43.160626356s) [noise=true , address=77.90.31.12:4604 , piece_id=6I7GIYQ2F2HFQMD77SKMPNJ6MZVVLDKBIAKP5IJAS72IPEUOJ7BQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390072439 (397h36m43.160713774s) [noise=true , address=77.90.31.16:4768 , piece_id=BDDAG4RMRGPY7XRPQUTSASCU4IM4IWSUFJMESKPICS7HPNYIZNBQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390278493 (397h36m43.160919828s) [noise=true , address=77.90.31.13:4642 , piece_id=J4PU4JOJHJSRJ7VVBTL6IZ6KIG53T7MV26ZKDXMC4KQGCHBLRTBQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390308300 (397h36m43.160949635s) [noise=true , address=77.90.31.19:4849 , piece_id=WCOWGUYSSMQBT7KGSTXUFHHBLERBO4Z4WPFVGQFNEU5BLETOREXA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390492182 (397h36m43.161133517s) [noise=true , address=79.127.219.38:4027 , piece_id=S4IHOISJGSNFG3X2WIVE4ICJQMRSE7B73VWYJNRCVTH5YU4OQBXQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390531768 (397h36m43.161173093s) [noise=true , address=104.36.82.23:4470 , piece_id=AHR7BOSC7B67VIEXLOZ5RUXN7OMSBQC4KRZEKR2D4XZ7BYRMDRXQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390672396 (397h36m43.161313732s) [noise=true , address=45.91.95.5:4024 , piece_id=MYVWP2IVYAMQNMKO2M4KXUVGPZQ5UGYDCEGSUU7XNQJ2NXTQ2ELQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390887989 (397h36m43.161529324s) [noise=true , address=45.91.95.4:4010 , piece_id=3DBDJ4TOP6TNDWO2WRLF7EA7C2TGBT2MDARICV5XNNUFEXTC37IA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:09.501089984 (397h36m44.271731319s) [upload=318117]: Next returned nil segment
Jul 9 22:47:09.501112097 (397h36m44.271753432s) [upload=318117]: Waiting for error group managing segments...
Jul 9 22:47:09.782624903 (397h36m44.553266238s) [noise=true , address=79.127.205.233:4629 , piece_id=SCMFOAOYWF3I6ICMJ75QAQROZJIQYISZBU6NHKV7NQ75YNRWNDWQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.388772974 (397h36m45.159414309s) [noise=true , address=77.90.31.17:4800 , piece_id=65FVQXPTFSDSBM7OF2HZLVPIQRUH3L4IA3VQTPQCSGYURTFP3CWA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.390566435 (397h36m45.16120777s) [noise=true , address=77.90.31.19:4849 , piece_id=WCOWGUYSSMQBT7KGSTXUFHHBLERBO4Z4WPFVGQFNEU5BLETOREXA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.393971279 (397h36m45.164612614s) [noise=true , address=77.90.31.13:4642 , piece_id=J4PU4JOJHJSRJ7VVBTL6IZ6KIG53T7MV26ZKDXMC4KQGCHBLRTBQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.394105566 (397h36m45.164746891s) [noise=true , address=77.90.31.12:4604 , piece_id=6I7GIYQ2F2HFQMD77SKMPNJ6MZVVLDKBIAKP5IJAS72IPEUOJ7BQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.401165262 (397h36m45.171806597s) [noise=true , address=77.90.31.16:4768 , piece_id=BDDAG4RMRGPY7XRPQUTSASCU4IM4IWSUFJMESKPICS7HPNYIZNBQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.404084396 (397h36m45.174725721s) [noise=true , address=77.90.31.11:4580 , piece_id=TY4475YRV6Z7BJ5GUQPNAVIPL3IDSTUGNSYQN3COP3YF7AAINKLA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:15.282126109 (397h36m50.052767444s) [noise=true , address=79.127.219.38:4027 , piece_id=S4IHOISJGSNFG3X2WIVE4ICJQMRSE7B73VWYJNRCVTH5YU4OQBXQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:21.052417283 (397h36m55.823058659s) [noise=true , address=104.36.82.60:4363 , piece_id=R3L3AUCM5G4NDXTEEVMA3LUMAPVO2RK6YPYCPLOVGLFHSOVE35YA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:23.506595216 (397h36m58.277236551s) [noise=true , address=104.36.82.23:4470 , piece_id=AHR7BOSC7B67VIEXLOZ5RUXN7OMSBQC4KRZEKR2D4XZ7BYRMDRXQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:38.125951632 (397h37m12.896592967s) [noise=true , address=104.36.82.38:4724 , piece_id=ZQVGYE67IKOQFNIIGKQA3VRGKIODUD6AQIM6CPMAE2MZEDPMRLJQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:46.326357492 (397h37m21.096998827s) [noise=true , address=104.36.82.19:4414 , piece_id=R4MZB3S2TVIGFQAPOOZVTRUWEAITFVMJ2BOD2ENGFISBZ6SQOXZQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:50.789365539 (397h37m25.560006874s) [noise=true , address=104.36.82.20:4453 , piece_id=53UPZW3K64NMBD2VWD3WFIDQUPL2D2HB67CUHMCXFAVTBCPPF3YQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:48:07.307076395 (397h37m42.07771772s) [noise=true , address=45.91.95.4:4010 , piece_id=3DBDJ4TOP6TNDWO2WRLF7EA7C2TGBT2MDARICV5XNNUFEXTC37IA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:129tyPaFUJBVtavq2kR9ZmPit8FpMMUNcQDaPxtSjbyucwze52G, address:45.91.95.4:4010): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307118365 (397h37m42.0777597s) [noise=true , address=45.91.95.3:4029 , piece_id=KIDHLHXLDN6ZFTABE6PFO26IU4SRF4XX3HUG556LDKA2IN5LIYMQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1tZENR6yxE22mBd53o8NbnUw5AYr4ncGSj91RTqYnsMv4EL4Yq, address:45.91.95.3:4029): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307124888 (397h37m42.077766223s) [noise=true , address=45.91.95.2:4002 , piece_id=V6WN7U6WI67BNTETWNJGST5TDPLR6BZHI5JM6MEVD27HVW6O3ERA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307150196 (397h37m42.077791532s) [noise=true , address=45.91.95.5:4024 , piece_id=MYVWP2IVYAMQNMKO2M4KXUVGPZQ5UGYDCEGSUU7XNQJ2NXTQ2ELQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1FJd9VQZBcEN3pv5totxoNNc3eMwpyUM4vbBsbgKAu6RfZeszf, address:45.91.95.5:4024): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307161688 (397h37m42.077803023s) [noise=true , address=104.36.82.36:4638 , piece_id=I4I6P7RDI2OC3Z5FJP6ZUWPSF3UNCMHVRZHEQPDRAI7YDREWM3OA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1poiPDhUGb5ZGnZnZcRViY45RefT9anD83YFcqZVZspnnJ9fN5, address:104.36.82.36:4638): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307181156 (397h37m42.077822491s) [seg_pos={0 0}, upload=318117]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 22:48:07.534569316 (397h37m42.305210681s) [upload=318117]: Done uploading segments...
Jul 9 22:48:07.534598401 (397h37m42.305239766s) [upload=318117]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 13.585460122s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..............X.....................X.......................] duration:
34.98898601s
FAIL (2025-07-09 22:49:03 UTC)
orbiter warning: context deadline exceeded
Duration: 34.989007521s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.X.X.............X.........................X...............X] duration:
30.001223745s
FAIL (2025-07-09 22:48:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001245076s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.X.X............X.........X...............................X.] duration:
16.415716922s
FAIL (2025-07-09 22:48:24 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
:03.248453150 (397h36m38.019094495s) [upload=318103]: Upload finished. err: <nil>
Jul 9 22:47:07.946019126 (397h36m42.716660461s) [upload=318117]: Starting upload
Jul 9 22:47:07.946077277 (397h36m42.716718612s) [upload=318117]: Done starting upload
Jul 9 22:47:07.946189321 (397h36m42.716830656s) [upload=318117]: Uploading segments...
Jul 9 22:47:07.946280796 (397h36m42.716922131s) [upload=318117]: Got next segment. Inline: false
Jul 9 22:47:07.946333867 (397h36m42.716975192s) [upload=318117]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:47:07.946338967 (397h36m42.716980302s) [upload=318117]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:47:08.388438504 (397h36m43.159079839s) [seg_pos={0 0}, upload=318117]: Begin upload segment...
Jul 9 22:47:08.388478570 (397h36m43.159119905s) [seg_pos={0 0}, upload=318117]: Done begin upload segment.
Jul 9 22:47:08.388502175 (397h36m43.15914351s) [noise=true , address=79.127.205.233:4629 , piece_id=SCMFOAOYWF3I6ICMJ75QAQROZJIQYISZBU6NHKV7NQ75YNRWNDWQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.388508116 (397h36m43.159149451s) [noise=true , address=104.36.82.36:4638 , piece_id=I4I6P7RDI2OC3Z5FJP6ZUWPSF3UNCMHVRZHEQPDRAI7YDREWM3OA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.388551971 (397h36m43.159193296s) [noise=true , address=45.91.95.3:4029 , piece_id=KIDHLHXLDN6ZFTABE6PFO26IU4SRF4XX3HUG556LDKA2IN5LIYMQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.388844890 (397h36m43.159486215s) [noise=true , address=104.36.82.20:4453 , piece_id=53UPZW3K64NMBD2VWD3WFIDQUPL2D2HB67CUHMCXFAVTBCPPF3YQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389067626 (397h36m43.159708951s) [noise=true , address=77.90.31.17:4800 , piece_id=65FVQXPTFSDSBM7OF2HZLVPIQRUH3L4IA3VQTPQCSGYURTFP3CWA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389087926 (397h36m43.159729251s) [noise=true , address=104.36.82.60:4363 , piece_id=R3L3AUCM5G4NDXTEEVMA3LUMAPVO2RK6YPYCPLOVGLFHSOVE35YA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389330299 (397h36m43.159971624s) [noise=true , address=104.36.82.38:4724 , piece_id=ZQVGYE67IKOQFNIIGKQA3VRGKIODUD6AQIM6CPMAE2MZEDPMRLJQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389524060 (397h36m43.160165395s) [noise=true , address=45.91.95.2:4002 , piece_id=V6WN7U6WI67BNTETWNJGST5TDPLR6BZHI5JM6MEVD27HVW6O3ERA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389567773 (397h36m43.160209098s) [noise=true , address=77.90.31.11:4580 , piece_id=TY4475YRV6Z7BJ5GUQPNAVIPL3IDSTUGNSYQN3COP3YF7AAINKLA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389825587 (397h36m43.160466922s) [noise=true , address=104.36.82.19:4414 , piece_id=R4MZB3S2TVIGFQAPOOZVTRUWEAITFVMJ2BOD2ENGFISBZ6SQOXZQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389985021 (397h36m43.160626356s) [noise=true , address=77.90.31.12:4604 , piece_id=6I7GIYQ2F2HFQMD77SKMPNJ6MZVVLDKBIAKP5IJAS72IPEUOJ7BQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390072439 (397h36m43.160713774s) [noise=true , address=77.90.31.16:4768 , piece_id=BDDAG4RMRGPY7XRPQUTSASCU4IM4IWSUFJMESKPICS7HPNYIZNBQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390278493 (397h36m43.160919828s) [noise=true , address=77.90.31.13:4642 , piece_id=J4PU4JOJHJSRJ7VVBTL6IZ6KIG53T7MV26ZKDXMC4KQGCHBLRTBQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390308300 (397h36m43.160949635s) [noise=true , address=77.90.31.19:4849 , piece_id=WCOWGUYSSMQBT7KGSTXUFHHBLERBO4Z4WPFVGQFNEU5BLETOREXA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390492182 (397h36m43.161133517s) [noise=true , address=79.127.219.38:4027 , piece_id=S4IHOISJGSNFG3X2WIVE4ICJQMRSE7B73VWYJNRCVTH5YU4OQBXQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390531768 (397h36m43.161173093s) [noise=true , address=104.36.82.23:4470 , piece_id=AHR7BOSC7B67VIEXLOZ5RUXN7OMSBQC4KRZEKR2D4XZ7BYRMDRXQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390672396 (397h36m43.161313732s) [noise=true , address=45.91.95.5:4024 , piece_id=MYVWP2IVYAMQNMKO2M4KXUVGPZQ5UGYDCEGSUU7XNQJ2NXTQ2ELQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390887989 (397h36m43.161529324s) [noise=true , address=45.91.95.4:4010 , piece_id=3DBDJ4TOP6TNDWO2WRLF7EA7C2TGBT2MDARICV5XNNUFEXTC37IA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:09.501089984 (397h36m44.271731319s) [upload=318117]: Next returned nil segment
Jul 9 22:47:09.501112097 (397h36m44.271753432s) [upload=318117]: Waiting for error group managing segments...
Jul 9 22:47:09.782624903 (397h36m44.553266238s) [noise=true , address=79.127.205.233:4629 , piece_id=SCMFOAOYWF3I6ICMJ75QAQROZJIQYISZBU6NHKV7NQ75YNRWNDWQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.388772974 (397h36m45.159414309s) [noise=true , address=77.90.31.17:4800 , piece_id=65FVQXPTFSDSBM7OF2HZLVPIQRUH3L4IA3VQTPQCSGYURTFP3CWA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.390566435 (397h36m45.16120777s) [noise=true , address=77.90.31.19:4849 , piece_id=WCOWGUYSSMQBT7KGSTXUFHHBLERBO4Z4WPFVGQFNEU5BLETOREXA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.393971279 (397h36m45.164612614s) [noise=true , address=77.90.31.13:4642 , piece_id=J4PU4JOJHJSRJ7VVBTL6IZ6KIG53T7MV26ZKDXMC4KQGCHBLRTBQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.394105566 (397h36m45.164746891s) [noise=true , address=77.90.31.12:4604 , piece_id=6I7GIYQ2F2HFQMD77SKMPNJ6MZVVLDKBIAKP5IJAS72IPEUOJ7BQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.401165262 (397h36m45.171806597s) [noise=true , address=77.90.31.16:4768 , piece_id=BDDAG4RMRGPY7XRPQUTSASCU4IM4IWSUFJMESKPICS7HPNYIZNBQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.404084396 (397h36m45.174725721s) [noise=true , address=77.90.31.11:4580 , piece_id=TY4475YRV6Z7BJ5GUQPNAVIPL3IDSTUGNSYQN3COP3YF7AAINKLA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:15.282126109 (397h36m50.052767444s) [noise=true , address=79.127.219.38:4027 , piece_id=S4IHOISJGSNFG3X2WIVE4ICJQMRSE7B73VWYJNRCVTH5YU4OQBXQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:21.052417283 (397h36m55.823058659s) [noise=true , address=104.36.82.60:4363 , piece_id=R3L3AUCM5G4NDXTEEVMA3LUMAPVO2RK6YPYCPLOVGLFHSOVE35YA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:23.506595216 (397h36m58.277236551s) [noise=true , address=104.36.82.23:4470 , piece_id=AHR7BOSC7B67VIEXLOZ5RUXN7OMSBQC4KRZEKR2D4XZ7BYRMDRXQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:38.125951632 (397h37m12.896592967s) [noise=true , address=104.36.82.38:4724 , piece_id=ZQVGYE67IKOQFNIIGKQA3VRGKIODUD6AQIM6CPMAE2MZEDPMRLJQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:46.326357492 (397h37m21.096998827s) [noise=true , address=104.36.82.19:4414 , piece_id=R4MZB3S2TVIGFQAPOOZVTRUWEAITFVMJ2BOD2ENGFISBZ6SQOXZQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:50.789365539 (397h37m25.560006874s) [noise=true , address=104.36.82.20:4453 , piece_id=53UPZW3K64NMBD2VWD3WFIDQUPL2D2HB67CUHMCXFAVTBCPPF3YQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:48:07.307076395 (397h37m42.07771772s) [noise=true , address=45.91.95.4:4010 , piece_id=3DBDJ4TOP6TNDWO2WRLF7EA7C2TGBT2MDARICV5XNNUFEXTC37IA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:129tyPaFUJBVtavq2kR9ZmPit8FpMMUNcQDaPxtSjbyucwze52G, address:45.91.95.4:4010): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307118365 (397h37m42.0777597s) [noise=true , address=45.91.95.3:4029 , piece_id=KIDHLHXLDN6ZFTABE6PFO26IU4SRF4XX3HUG556LDKA2IN5LIYMQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1tZENR6yxE22mBd53o8NbnUw5AYr4ncGSj91RTqYnsMv4EL4Yq, address:45.91.95.3:4029): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307124888 (397h37m42.077766223s) [noise=true , address=45.91.95.2:4002 , piece_id=V6WN7U6WI67BNTETWNJGST5TDPLR6BZHI5JM6MEVD27HVW6O3ERA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307150196 (397h37m42.077791532s) [noise=true , address=45.91.95.5:4024 , piece_id=MYVWP2IVYAMQNMKO2M4KXUVGPZQ5UGYDCEGSUU7XNQJ2NXTQ2ELQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1FJd9VQZBcEN3pv5totxoNNc3eMwpyUM4vbBsbgKAu6RfZeszf, address:45.91.95.5:4024): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307161688 (397h37m42.077803023s) [noise=true , address=104.36.82.36:4638 , piece_id=I4I6P7RDI2OC3Z5FJP6ZUWPSF3UNCMHVRZHEQPDRAI7YDREWM3OA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1poiPDhUGb5ZGnZnZcRViY45RefT9anD83YFcqZVZspnnJ9fN5, address:104.36.82.36:4638): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307181156 (397h37m42.077822491s) [seg_pos={0 0}, upload=318117]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 22:48:07.534569316 (397h37m42.305210681s) [upload=318117]: Done uploading segments...
Jul 9 22:48:07.534598401 (397h37m42.305239766s) [upload=318117]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 16.415734356s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [..X.X............X.........X...............................X] duration:
1m0.229116547s
FAIL (2025-07-09 22:48:07 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
:03.248453150 (397h36m38.019094495s) [upload=318103]: Upload finished. err: <nil>
Jul 9 22:47:07.946019126 (397h36m42.716660461s) [upload=318117]: Starting upload
Jul 9 22:47:07.946077277 (397h36m42.716718612s) [upload=318117]: Done starting upload
Jul 9 22:47:07.946189321 (397h36m42.716830656s) [upload=318117]: Uploading segments...
Jul 9 22:47:07.946280796 (397h36m42.716922131s) [upload=318117]: Got next segment. Inline: false
Jul 9 22:47:07.946333867 (397h36m42.716975192s) [upload=318117]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:47:07.946338967 (397h36m42.716980302s) [upload=318117]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:47:08.388438504 (397h36m43.159079839s) [seg_pos={0 0}, upload=318117]: Begin upload segment...
Jul 9 22:47:08.388478570 (397h36m43.159119905s) [seg_pos={0 0}, upload=318117]: Done begin upload segment.
Jul 9 22:47:08.388502175 (397h36m43.15914351s) [noise=true , address=79.127.205.233:4629 , piece_id=SCMFOAOYWF3I6ICMJ75QAQROZJIQYISZBU6NHKV7NQ75YNRWNDWQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.388508116 (397h36m43.159149451s) [noise=true , address=104.36.82.36:4638 , piece_id=I4I6P7RDI2OC3Z5FJP6ZUWPSF3UNCMHVRZHEQPDRAI7YDREWM3OA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.388551971 (397h36m43.159193296s) [noise=true , address=45.91.95.3:4029 , piece_id=KIDHLHXLDN6ZFTABE6PFO26IU4SRF4XX3HUG556LDKA2IN5LIYMQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.388844890 (397h36m43.159486215s) [noise=true , address=104.36.82.20:4453 , piece_id=53UPZW3K64NMBD2VWD3WFIDQUPL2D2HB67CUHMCXFAVTBCPPF3YQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389067626 (397h36m43.159708951s) [noise=true , address=77.90.31.17:4800 , piece_id=65FVQXPTFSDSBM7OF2HZLVPIQRUH3L4IA3VQTPQCSGYURTFP3CWA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389087926 (397h36m43.159729251s) [noise=true , address=104.36.82.60:4363 , piece_id=R3L3AUCM5G4NDXTEEVMA3LUMAPVO2RK6YPYCPLOVGLFHSOVE35YA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389330299 (397h36m43.159971624s) [noise=true , address=104.36.82.38:4724 , piece_id=ZQVGYE67IKOQFNIIGKQA3VRGKIODUD6AQIM6CPMAE2MZEDPMRLJQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389524060 (397h36m43.160165395s) [noise=true , address=45.91.95.2:4002 , piece_id=V6WN7U6WI67BNTETWNJGST5TDPLR6BZHI5JM6MEVD27HVW6O3ERA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389567773 (397h36m43.160209098s) [noise=true , address=77.90.31.11:4580 , piece_id=TY4475YRV6Z7BJ5GUQPNAVIPL3IDSTUGNSYQN3COP3YF7AAINKLA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389825587 (397h36m43.160466922s) [noise=true , address=104.36.82.19:4414 , piece_id=R4MZB3S2TVIGFQAPOOZVTRUWEAITFVMJ2BOD2ENGFISBZ6SQOXZQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.389985021 (397h36m43.160626356s) [noise=true , address=77.90.31.12:4604 , piece_id=6I7GIYQ2F2HFQMD77SKMPNJ6MZVVLDKBIAKP5IJAS72IPEUOJ7BQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390072439 (397h36m43.160713774s) [noise=true , address=77.90.31.16:4768 , piece_id=BDDAG4RMRGPY7XRPQUTSASCU4IM4IWSUFJMESKPICS7HPNYIZNBQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390278493 (397h36m43.160919828s) [noise=true , address=77.90.31.13:4642 , piece_id=J4PU4JOJHJSRJ7VVBTL6IZ6KIG53T7MV26ZKDXMC4KQGCHBLRTBQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390308300 (397h36m43.160949635s) [noise=true , address=77.90.31.19:4849 , piece_id=WCOWGUYSSMQBT7KGSTXUFHHBLERBO4Z4WPFVGQFNEU5BLETOREXA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390492182 (397h36m43.161133517s) [noise=true , address=79.127.219.38:4027 , piece_id=S4IHOISJGSNFG3X2WIVE4ICJQMRSE7B73VWYJNRCVTH5YU4OQBXQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390531768 (397h36m43.161173093s) [noise=true , address=104.36.82.23:4470 , piece_id=AHR7BOSC7B67VIEXLOZ5RUXN7OMSBQC4KRZEKR2D4XZ7BYRMDRXQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390672396 (397h36m43.161313732s) [noise=true , address=45.91.95.5:4024 , piece_id=MYVWP2IVYAMQNMKO2M4KXUVGPZQ5UGYDCEGSUU7XNQJ2NXTQ2ELQ, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:08.390887989 (397h36m43.161529324s) [noise=true , address=45.91.95.4:4010 , piece_id=3DBDJ4TOP6TNDWO2WRLF7EA7C2TGBT2MDARICV5XNNUFEXTC37IA, seg_pos={0 0}, upload=318117]: Uploading piece...
Jul 9 22:47:09.501089984 (397h36m44.271731319s) [upload=318117]: Next returned nil segment
Jul 9 22:47:09.501112097 (397h36m44.271753432s) [upload=318117]: Waiting for error group managing segments...
Jul 9 22:47:09.782624903 (397h36m44.553266238s) [noise=true , address=79.127.205.233:4629 , piece_id=SCMFOAOYWF3I6ICMJ75QAQROZJIQYISZBU6NHKV7NQ75YNRWNDWQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.388772974 (397h36m45.159414309s) [noise=true , address=77.90.31.17:4800 , piece_id=65FVQXPTFSDSBM7OF2HZLVPIQRUH3L4IA3VQTPQCSGYURTFP3CWA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.390566435 (397h36m45.16120777s) [noise=true , address=77.90.31.19:4849 , piece_id=WCOWGUYSSMQBT7KGSTXUFHHBLERBO4Z4WPFVGQFNEU5BLETOREXA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.393971279 (397h36m45.164612614s) [noise=true , address=77.90.31.13:4642 , piece_id=J4PU4JOJHJSRJ7VVBTL6IZ6KIG53T7MV26ZKDXMC4KQGCHBLRTBQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.394105566 (397h36m45.164746891s) [noise=true , address=77.90.31.12:4604 , piece_id=6I7GIYQ2F2HFQMD77SKMPNJ6MZVVLDKBIAKP5IJAS72IPEUOJ7BQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.401165262 (397h36m45.171806597s) [noise=true , address=77.90.31.16:4768 , piece_id=BDDAG4RMRGPY7XRPQUTSASCU4IM4IWSUFJMESKPICS7HPNYIZNBQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:10.404084396 (397h36m45.174725721s) [noise=true , address=77.90.31.11:4580 , piece_id=TY4475YRV6Z7BJ5GUQPNAVIPL3IDSTUGNSYQN3COP3YF7AAINKLA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:15.282126109 (397h36m50.052767444s) [noise=true , address=79.127.219.38:4027 , piece_id=S4IHOISJGSNFG3X2WIVE4ICJQMRSE7B73VWYJNRCVTH5YU4OQBXQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:21.052417283 (397h36m55.823058659s) [noise=true , address=104.36.82.60:4363 , piece_id=R3L3AUCM5G4NDXTEEVMA3LUMAPVO2RK6YPYCPLOVGLFHSOVE35YA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:23.506595216 (397h36m58.277236551s) [noise=true , address=104.36.82.23:4470 , piece_id=AHR7BOSC7B67VIEXLOZ5RUXN7OMSBQC4KRZEKR2D4XZ7BYRMDRXQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:38.125951632 (397h37m12.896592967s) [noise=true , address=104.36.82.38:4724 , piece_id=ZQVGYE67IKOQFNIIGKQA3VRGKIODUD6AQIM6CPMAE2MZEDPMRLJQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:46.326357492 (397h37m21.096998827s) [noise=true , address=104.36.82.19:4414 , piece_id=R4MZB3S2TVIGFQAPOOZVTRUWEAITFVMJ2BOD2ENGFISBZ6SQOXZQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:47:50.789365539 (397h37m25.560006874s) [noise=true , address=104.36.82.20:4453 , piece_id=53UPZW3K64NMBD2VWD3WFIDQUPL2D2HB67CUHMCXFAVTBCPPF3YQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: <nil>
Jul 9 22:48:07.307076395 (397h37m42.07771772s) [noise=true , address=45.91.95.4:4010 , piece_id=3DBDJ4TOP6TNDWO2WRLF7EA7C2TGBT2MDARICV5XNNUFEXTC37IA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:129tyPaFUJBVtavq2kR9ZmPit8FpMMUNcQDaPxtSjbyucwze52G, address:45.91.95.4:4010): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307118365 (397h37m42.0777597s) [noise=true , address=45.91.95.3:4029 , piece_id=KIDHLHXLDN6ZFTABE6PFO26IU4SRF4XX3HUG556LDKA2IN5LIYMQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1tZENR6yxE22mBd53o8NbnUw5AYr4ncGSj91RTqYnsMv4EL4Yq, address:45.91.95.3:4029): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307124888 (397h37m42.077766223s) [noise=true , address=45.91.95.2:4002 , piece_id=V6WN7U6WI67BNTETWNJGST5TDPLR6BZHI5JM6MEVD27HVW6O3ERA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307150196 (397h37m42.077791532s) [noise=true , address=45.91.95.5:4024 , piece_id=MYVWP2IVYAMQNMKO2M4KXUVGPZQ5UGYDCEGSUU7XNQJ2NXTQ2ELQ, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1FJd9VQZBcEN3pv5totxoNNc3eMwpyUM4vbBsbgKAu6RfZeszf, address:45.91.95.5:4024): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307161688 (397h37m42.077803023s) [noise=true , address=104.36.82.36:4638 , piece_id=I4I6P7RDI2OC3Z5FJP6ZUWPSF3UNCMHVRZHEQPDRAI7YDREWM3OA, seg_pos={0 0}, upload=318117]: Done uploading piece. err: ecclient: upload failed (node:1poiPDhUGb5ZGnZnZcRViY45RefT9anD83YFcqZVZspnnJ9fN5, address:104.36.82.36:4638): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:48:07.307181156 (397h37m42.077822491s) [seg_pos={0 0}, upload=318117]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 22:48:07.534569316 (397h37m42.305210681s) [upload=318117]: Done uploading segments...
Jul 9 22:48:07.534598401 (397h37m42.305239766s) [upload=318117]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 1m0.229130183s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
4.932262186s
FAIL (2025-07-09 22:48:05 UTC)
orbiter warning: uplink: encryption: context deadline exceeded
Duration: 4.932287214s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...............X.....................X......................] duration:
34.649681305s
FAIL (2025-07-09 22:47:50 UTC)
orbiter warning: context deadline exceeded
Duration: 34.649692446s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [................X.....................X.....................] duration:
38.419328771s
FAIL (2025-07-09 22:47:15 UTC)
orbiter warning: context deadline exceeded
Duration: 38.419341486s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...X.X............X.........X...............................] duration:
47.534401777s
FAIL (2025-07-09 22:47:07 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
:27.530694236 (397h12m2.301335571s) [upload=317782]: Got next segment. Inline: false
Jul 9 22:22:27.530721418 (397h12m2.301362743s) [upload=317782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:22:27.530725656 (397h12m2.301366981s) [upload=317782]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:22:27.972697777 (397h12m2.743339102s) [seg_pos={0 0}, upload=317782]: Begin upload segment...
Jul 9 22:22:27.972768972 (397h12m2.743410308s) [seg_pos={0 0}, upload=317782]: Done begin upload segment.
Jul 9 22:22:27.972793770 (397h12m2.743435105s) [noise=true , address=77.90.31.13:4649 , piece_id=NYXVP7IV6MQBMUWGECBVUUZ3RYF5XQUXTBMJQFPX3GPA2E4FMTEQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973170050 (397h12m2.743811385s) [noise=true , address=77.90.31.16:4749 , piece_id=IUTCAJL2PVYOLFP33YMWX2B5XP2D2FYVR6UBGKNC7QL6OHCMRL6Q, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973207692 (397h12m2.743849027s) [noise=true , address=104.36.82.19:4429 , piece_id=T6LIMFSHHRXZMQYYXDFTZVZ3GATOMH5LGHCCDLD4MFHK5XU2PSSQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973462760 (397h12m2.744104095s) [noise=true , address=45.91.116.10:4090 , piece_id=PGHOH3Q7RSNSEOC6QAELKNK6RR5GFURWMOGXEBX7DZ64WXNRD4IA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973561820 (397h12m2.744203155s) [noise=true , address=79.127.220.97:4371 , piece_id=7C6V6Z6P66SYVSA2NSHUKHPVIV22RS4PASY26INJYPF5DGDDABXA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973719421 (397h12m2.744360756s) [noise=true , address=77.90.31.12:4594 , piece_id=LJJXKPHBZKSNCJTOQFDOYBTL4U7ZBSE7RI57EWMI5CZJ5YD5YI3A, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973862605 (397h12m2.74450394s) [noise=true , address=77.90.31.17:4781 , piece_id=LCSMEGJZ35OL7TEOKSTMPROLECXWA6MXWDM3QY3GQZ3LBFA4C2UQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973978998 (397h12m2.744620333s) [noise=true , address=45.91.95.5:4019 , piece_id=WQBPU5OUOFBIT6W6NAUTNFMFI3UU7MBK64U3UET7HN4X7TMCYY3A, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974101712 (397h12m2.744743047s) [noise=true , address=104.36.82.35:4593 , piece_id=CP5TQJXAV2OWWXENMFF44P5DI5QU5HIYWAXH525ITTIR4NDTZHLA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974232022 (397h12m2.744873357s) [noise=true , address=45.91.95.3:4003 , piece_id=OEBEE652D53YZZYWIWIKA7GFVBFIJ2FR5VMNOA7HKNNFWOQBDHVQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974306995 (397h12m2.74494833s) [noise=true , address=104.36.82.3:4774 , piece_id=4EJYB2DWB736CXZ7CTQOE442AZWQJ2TCDTMMCMKXSANWBUIIABLA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974500055 (397h12m2.74514139s) [noise=true , address=77.90.31.21:4894 , piece_id=56U7WC7KQ2S7XLBDNVI7ADMZQX552THVHVK4DANCJF5FJ4XU7TOA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974522848 (397h12m2.745164183s) [noise=true , address=45.91.95.2:4002 , piece_id=N4RIOYFXEIYYJ2DOO3GRRGLPUBNSFDG2OHNXC6TMSYH2KX3TUWXQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974681611 (397h12m2.745322946s) [noise=true , address=104.36.82.18:4369 , piece_id=7EW62VOKMH6CXC4CURNXOAS65EXFSIARSR7MXLT2MAS2SJGQWFHQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974766505 (397h12m2.74540783s) [noise=true , address=77.90.31.19:4859 , piece_id=SZQ4CMWGIVVHARJOMKKTN6DI7VAZGVJQPXZKYCTUPPRVU5ZNGCVQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974923444 (397h12m2.745564779s) [noise=true , address=77.90.31.15:4701 , piece_id=T42LFWTYWBBQFFDYHVU6U5YO2H3RUKIJR2DI3U4FSXY4WEU5ZTFA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.975010481 (397h12m2.745651816s) [noise=true , address=77.90.31.14:4690 , piece_id=XT6BVPU5ZEAGNEPCTPXD7D35GBMUHGD2OGGEY2P22QIQM4EAYELQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.975147763 (397h12m2.745789088s) [noise=true , address=104.36.82.17:4345 , piece_id=QXC4VIW4NJ5376SZZLSLG62CGH4YUSHO77UQPGFVKBHKALFX6XOQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:29.327909891 (397h12m4.098551226s) [upload=317782]: Next returned nil segment
Jul 9 22:22:29.327933787 (397h12m4.098575122s) [upload=317782]: Waiting for error group managing segments...
Jul 9 22:22:29.978258832 (397h12m4.748900167s) [noise=true , address=77.90.31.12:4594 , piece_id=LJJXKPHBZKSNCJTOQFDOYBTL4U7ZBSE7RI57EWMI5CZJ5YD5YI3A, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.979151280 (397h12m4.749792615s) [noise=true , address=77.90.31.21:4894 , piece_id=56U7WC7KQ2S7XLBDNVI7ADMZQX552THVHVK4DANCJF5FJ4XU7TOA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.984195199 (397h12m4.754836534s) [noise=true , address=77.90.31.19:4859 , piece_id=SZQ4CMWGIVVHARJOMKKTN6DI7VAZGVJQPXZKYCTUPPRVU5ZNGCVQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.988046737 (397h12m4.758688072s) [noise=true , address=77.90.31.15:4701 , piece_id=T42LFWTYWBBQFFDYHVU6U5YO2H3RUKIJR2DI3U4FSXY4WEU5ZTFA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.992300966 (397h12m4.762942301s) [noise=true , address=77.90.31.16:4749 , piece_id=IUTCAJL2PVYOLFP33YMWX2B5XP2D2FYVR6UBGKNC7QL6OHCMRL6Q, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.996959930 (397h12m4.767601265s) [noise=true , address=77.90.31.17:4781 , piece_id=LCSMEGJZ35OL7TEOKSTMPROLECXWA6MXWDM3QY3GQZ3LBFA4C2UQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.997648557 (397h12m4.768289892s) [noise=true , address=77.90.31.14:4690 , piece_id=XT6BVPU5ZEAGNEPCTPXD7D35GBMUHGD2OGGEY2P22QIQM4EAYELQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.008095954 (397h12m4.778737289s) [noise=true , address=45.91.116.10:4090 , piece_id=PGHOH3Q7RSNSEOC6QAELKNK6RR5GFURWMOGXEBX7DZ64WXNRD4IA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.008404595 (397h12m4.77904593s) [noise=true , address=77.90.31.13:4649 , piece_id=NYXVP7IV6MQBMUWGECBVUUZ3RYF5XQUXTBMJQFPX3GPA2E4FMTEQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.392350092 (397h12m5.162991417s) [noise=true , address=79.127.220.97:4371 , piece_id=7C6V6Z6P66SYVSA2NSHUKHPVIV22RS4PASY26INJYPF5DGDDABXA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:24.652397067 (397h12m59.423038402s) [noise=true , address=104.36.82.17:4345 , piece_id=QXC4VIW4NJ5376SZZLSLG62CGH4YUSHO77UQPGFVKBHKALFX6XOQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:26.282002097 (397h13m1.052643432s) [noise=true , address=104.36.82.35:4593 , piece_id=CP5TQJXAV2OWWXENMFF44P5DI5QU5HIYWAXH525ITTIR4NDTZHLA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:26.892929169 (397h13m1.663570504s) [noise=true , address=45.91.95.5:4019 , piece_id=WQBPU5OUOFBIT6W6NAUTNFMFI3UU7MBK64U3UET7HN4X7TMCYY3A, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:45.91.95.5:4019): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.892983052 (397h13m1.663624377s) [noise=true , address=104.36.82.19:4429 , piece_id=T6LIMFSHHRXZMQYYXDFTZVZ3GATOMH5LGHCCDLD4MFHK5XU2PSSQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1Bi1qafYFw9NukFHUJzV9M4Jd1i7YsjQMs7jXsVUn44pecVWzr, address:104.36.82.19:4429): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893020182 (397h13m1.663661517s) [noise=true , address=45.91.95.3:4003 , piece_id=OEBEE652D53YZZYWIWIKA7GFVBFIJ2FR5VMNOA7HKNNFWOQBDHVQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:12775mgnBsA3595WztdcNsUBVWCLeeZECij1L2EnwuvzCCtMouk, address:45.91.95.3:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893081459 (397h13m1.663722784s) [noise=true , address=104.36.82.3:4774 , piece_id=4EJYB2DWB736CXZ7CTQOE442AZWQJ2TCDTMMCMKXSANWBUIIABLA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1r3Ty7Q8v3b4nT3cVVzQqGDFpHdBu1XKPQvBy221Y78hPzv4Y9, address:104.36.82.3:4774): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893095246 (397h13m1.663736581s) [noise=true , address=104.36.82.18:4369 , piece_id=7EW62VOKMH6CXC4CURNXOAS65EXFSIARSR7MXLT2MAS2SJGQWFHQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1nwygPHyRJEZkjCoSZJTVhXDKLHc7AonVMzKMht6ppwZwhcGCT, address:104.36.82.18:4369): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893127879 (397h13m1.663769214s) [noise=true , address=45.91.95.2:4002 , piece_id=N4RIOYFXEIYYJ2DOO3GRRGLPUBNSFDG2OHNXC6TMSYH2KX3TUWXQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection; write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection
Jul 9 22:23:26.893140864 (397h13m1.663782199s) [seg_pos={0 0}, upload=317782]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Jul 9 22:23:27.126246135 (397h13m1.89688746s) [upload=317782]: Done uploading segments...
Jul 9 22:23:27.126270812 (397h13m1.896912147s) [upload=317782]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 47.534442103s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [..X.X.............X.........................X...............] duration:
6.402284013s
FAIL (2025-07-09 22:46:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.402299803s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
4.767663938s
FAIL (2025-07-09 22:46:51 UTC)
orbiter warning: uplink: encryption: context deadline exceeded
Duration: 4.767691892s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [...X.X.............X.........................X..............] duration:
6.218112398s
FAIL (2025-07-09 22:46:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.218132496s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [....X.X............X.........X..............................] duration:
34.02025577s
FAIL (2025-07-09 22:46:19 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
:27.530694236 (397h12m2.301335571s) [upload=317782]: Got next segment. Inline: false
Jul 9 22:22:27.530721418 (397h12m2.301362743s) [upload=317782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:22:27.530725656 (397h12m2.301366981s) [upload=317782]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:22:27.972697777 (397h12m2.743339102s) [seg_pos={0 0}, upload=317782]: Begin upload segment...
Jul 9 22:22:27.972768972 (397h12m2.743410308s) [seg_pos={0 0}, upload=317782]: Done begin upload segment.
Jul 9 22:22:27.972793770 (397h12m2.743435105s) [noise=true , address=77.90.31.13:4649 , piece_id=NYXVP7IV6MQBMUWGECBVUUZ3RYF5XQUXTBMJQFPX3GPA2E4FMTEQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973170050 (397h12m2.743811385s) [noise=true , address=77.90.31.16:4749 , piece_id=IUTCAJL2PVYOLFP33YMWX2B5XP2D2FYVR6UBGKNC7QL6OHCMRL6Q, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973207692 (397h12m2.743849027s) [noise=true , address=104.36.82.19:4429 , piece_id=T6LIMFSHHRXZMQYYXDFTZVZ3GATOMH5LGHCCDLD4MFHK5XU2PSSQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973462760 (397h12m2.744104095s) [noise=true , address=45.91.116.10:4090 , piece_id=PGHOH3Q7RSNSEOC6QAELKNK6RR5GFURWMOGXEBX7DZ64WXNRD4IA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973561820 (397h12m2.744203155s) [noise=true , address=79.127.220.97:4371 , piece_id=7C6V6Z6P66SYVSA2NSHUKHPVIV22RS4PASY26INJYPF5DGDDABXA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973719421 (397h12m2.744360756s) [noise=true , address=77.90.31.12:4594 , piece_id=LJJXKPHBZKSNCJTOQFDOYBTL4U7ZBSE7RI57EWMI5CZJ5YD5YI3A, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973862605 (397h12m2.74450394s) [noise=true , address=77.90.31.17:4781 , piece_id=LCSMEGJZ35OL7TEOKSTMPROLECXWA6MXWDM3QY3GQZ3LBFA4C2UQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973978998 (397h12m2.744620333s) [noise=true , address=45.91.95.5:4019 , piece_id=WQBPU5OUOFBIT6W6NAUTNFMFI3UU7MBK64U3UET7HN4X7TMCYY3A, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974101712 (397h12m2.744743047s) [noise=true , address=104.36.82.35:4593 , piece_id=CP5TQJXAV2OWWXENMFF44P5DI5QU5HIYWAXH525ITTIR4NDTZHLA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974232022 (397h12m2.744873357s) [noise=true , address=45.91.95.3:4003 , piece_id=OEBEE652D53YZZYWIWIKA7GFVBFIJ2FR5VMNOA7HKNNFWOQBDHVQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974306995 (397h12m2.74494833s) [noise=true , address=104.36.82.3:4774 , piece_id=4EJYB2DWB736CXZ7CTQOE442AZWQJ2TCDTMMCMKXSANWBUIIABLA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974500055 (397h12m2.74514139s) [noise=true , address=77.90.31.21:4894 , piece_id=56U7WC7KQ2S7XLBDNVI7ADMZQX552THVHVK4DANCJF5FJ4XU7TOA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974522848 (397h12m2.745164183s) [noise=true , address=45.91.95.2:4002 , piece_id=N4RIOYFXEIYYJ2DOO3GRRGLPUBNSFDG2OHNXC6TMSYH2KX3TUWXQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974681611 (397h12m2.745322946s) [noise=true , address=104.36.82.18:4369 , piece_id=7EW62VOKMH6CXC4CURNXOAS65EXFSIARSR7MXLT2MAS2SJGQWFHQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974766505 (397h12m2.74540783s) [noise=true , address=77.90.31.19:4859 , piece_id=SZQ4CMWGIVVHARJOMKKTN6DI7VAZGVJQPXZKYCTUPPRVU5ZNGCVQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974923444 (397h12m2.745564779s) [noise=true , address=77.90.31.15:4701 , piece_id=T42LFWTYWBBQFFDYHVU6U5YO2H3RUKIJR2DI3U4FSXY4WEU5ZTFA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.975010481 (397h12m2.745651816s) [noise=true , address=77.90.31.14:4690 , piece_id=XT6BVPU5ZEAGNEPCTPXD7D35GBMUHGD2OGGEY2P22QIQM4EAYELQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.975147763 (397h12m2.745789088s) [noise=true , address=104.36.82.17:4345 , piece_id=QXC4VIW4NJ5376SZZLSLG62CGH4YUSHO77UQPGFVKBHKALFX6XOQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:29.327909891 (397h12m4.098551226s) [upload=317782]: Next returned nil segment
Jul 9 22:22:29.327933787 (397h12m4.098575122s) [upload=317782]: Waiting for error group managing segments...
Jul 9 22:22:29.978258832 (397h12m4.748900167s) [noise=true , address=77.90.31.12:4594 , piece_id=LJJXKPHBZKSNCJTOQFDOYBTL4U7ZBSE7RI57EWMI5CZJ5YD5YI3A, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.979151280 (397h12m4.749792615s) [noise=true , address=77.90.31.21:4894 , piece_id=56U7WC7KQ2S7XLBDNVI7ADMZQX552THVHVK4DANCJF5FJ4XU7TOA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.984195199 (397h12m4.754836534s) [noise=true , address=77.90.31.19:4859 , piece_id=SZQ4CMWGIVVHARJOMKKTN6DI7VAZGVJQPXZKYCTUPPRVU5ZNGCVQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.988046737 (397h12m4.758688072s) [noise=true , address=77.90.31.15:4701 , piece_id=T42LFWTYWBBQFFDYHVU6U5YO2H3RUKIJR2DI3U4FSXY4WEU5ZTFA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.992300966 (397h12m4.762942301s) [noise=true , address=77.90.31.16:4749 , piece_id=IUTCAJL2PVYOLFP33YMWX2B5XP2D2FYVR6UBGKNC7QL6OHCMRL6Q, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.996959930 (397h12m4.767601265s) [noise=true , address=77.90.31.17:4781 , piece_id=LCSMEGJZ35OL7TEOKSTMPROLECXWA6MXWDM3QY3GQZ3LBFA4C2UQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.997648557 (397h12m4.768289892s) [noise=true , address=77.90.31.14:4690 , piece_id=XT6BVPU5ZEAGNEPCTPXD7D35GBMUHGD2OGGEY2P22QIQM4EAYELQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.008095954 (397h12m4.778737289s) [noise=true , address=45.91.116.10:4090 , piece_id=PGHOH3Q7RSNSEOC6QAELKNK6RR5GFURWMOGXEBX7DZ64WXNRD4IA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.008404595 (397h12m4.77904593s) [noise=true , address=77.90.31.13:4649 , piece_id=NYXVP7IV6MQBMUWGECBVUUZ3RYF5XQUXTBMJQFPX3GPA2E4FMTEQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.392350092 (397h12m5.162991417s) [noise=true , address=79.127.220.97:4371 , piece_id=7C6V6Z6P66SYVSA2NSHUKHPVIV22RS4PASY26INJYPF5DGDDABXA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:24.652397067 (397h12m59.423038402s) [noise=true , address=104.36.82.17:4345 , piece_id=QXC4VIW4NJ5376SZZLSLG62CGH4YUSHO77UQPGFVKBHKALFX6XOQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:26.282002097 (397h13m1.052643432s) [noise=true , address=104.36.82.35:4593 , piece_id=CP5TQJXAV2OWWXENMFF44P5DI5QU5HIYWAXH525ITTIR4NDTZHLA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:26.892929169 (397h13m1.663570504s) [noise=true , address=45.91.95.5:4019 , piece_id=WQBPU5OUOFBIT6W6NAUTNFMFI3UU7MBK64U3UET7HN4X7TMCYY3A, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:45.91.95.5:4019): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.892983052 (397h13m1.663624377s) [noise=true , address=104.36.82.19:4429 , piece_id=T6LIMFSHHRXZMQYYXDFTZVZ3GATOMH5LGHCCDLD4MFHK5XU2PSSQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1Bi1qafYFw9NukFHUJzV9M4Jd1i7YsjQMs7jXsVUn44pecVWzr, address:104.36.82.19:4429): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893020182 (397h13m1.663661517s) [noise=true , address=45.91.95.3:4003 , piece_id=OEBEE652D53YZZYWIWIKA7GFVBFIJ2FR5VMNOA7HKNNFWOQBDHVQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:12775mgnBsA3595WztdcNsUBVWCLeeZECij1L2EnwuvzCCtMouk, address:45.91.95.3:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893081459 (397h13m1.663722784s) [noise=true , address=104.36.82.3:4774 , piece_id=4EJYB2DWB736CXZ7CTQOE442AZWQJ2TCDTMMCMKXSANWBUIIABLA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1r3Ty7Q8v3b4nT3cVVzQqGDFpHdBu1XKPQvBy221Y78hPzv4Y9, address:104.36.82.3:4774): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893095246 (397h13m1.663736581s) [noise=true , address=104.36.82.18:4369 , piece_id=7EW62VOKMH6CXC4CURNXOAS65EXFSIARSR7MXLT2MAS2SJGQWFHQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1nwygPHyRJEZkjCoSZJTVhXDKLHc7AonVMzKMht6ppwZwhcGCT, address:104.36.82.18:4369): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893127879 (397h13m1.663769214s) [noise=true , address=45.91.95.2:4002 , piece_id=N4RIOYFXEIYYJ2DOO3GRRGLPUBNSFDG2OHNXC6TMSYH2KX3TUWXQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection; write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection
Jul 9 22:23:26.893140864 (397h13m1.663782199s) [seg_pos={0 0}, upload=317782]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Jul 9 22:23:27.126246135 (397h13m1.89688746s) [upload=317782]: Done uploading segments...
Jul 9 22:23:27.126270812 (397h13m1.896912147s) [upload=317782]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 34.020273556s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
4.855639515s
FAIL (2025-07-09 22:45:56 UTC)
orbiter warning: uplink: encryption: context deadline exceeded
Duration: 4.855665374s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [....X.X.............X.........................X.............] duration:
6.157751097s
FAIL (2025-07-09 22:45:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.157775191s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [X................X.....................X....................] duration:
9.151652131s
FAIL (2025-07-09 22:45:36 UTC)
orbiter warning: context deadline exceeded
Duration: 9.15167159s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.....X.X............X.........X.............................] duration:
17.222328344s
FAIL (2025-07-09 22:45:22 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
:27.530694236 (397h12m2.301335571s) [upload=317782]: Got next segment. Inline: false
Jul 9 22:22:27.530721418 (397h12m2.301362743s) [upload=317782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:22:27.530725656 (397h12m2.301366981s) [upload=317782]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:22:27.972697777 (397h12m2.743339102s) [seg_pos={0 0}, upload=317782]: Begin upload segment...
Jul 9 22:22:27.972768972 (397h12m2.743410308s) [seg_pos={0 0}, upload=317782]: Done begin upload segment.
Jul 9 22:22:27.972793770 (397h12m2.743435105s) [noise=true , address=77.90.31.13:4649 , piece_id=NYXVP7IV6MQBMUWGECBVUUZ3RYF5XQUXTBMJQFPX3GPA2E4FMTEQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973170050 (397h12m2.743811385s) [noise=true , address=77.90.31.16:4749 , piece_id=IUTCAJL2PVYOLFP33YMWX2B5XP2D2FYVR6UBGKNC7QL6OHCMRL6Q, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973207692 (397h12m2.743849027s) [noise=true , address=104.36.82.19:4429 , piece_id=T6LIMFSHHRXZMQYYXDFTZVZ3GATOMH5LGHCCDLD4MFHK5XU2PSSQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973462760 (397h12m2.744104095s) [noise=true , address=45.91.116.10:4090 , piece_id=PGHOH3Q7RSNSEOC6QAELKNK6RR5GFURWMOGXEBX7DZ64WXNRD4IA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973561820 (397h12m2.744203155s) [noise=true , address=79.127.220.97:4371 , piece_id=7C6V6Z6P66SYVSA2NSHUKHPVIV22RS4PASY26INJYPF5DGDDABXA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973719421 (397h12m2.744360756s) [noise=true , address=77.90.31.12:4594 , piece_id=LJJXKPHBZKSNCJTOQFDOYBTL4U7ZBSE7RI57EWMI5CZJ5YD5YI3A, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973862605 (397h12m2.74450394s) [noise=true , address=77.90.31.17:4781 , piece_id=LCSMEGJZ35OL7TEOKSTMPROLECXWA6MXWDM3QY3GQZ3LBFA4C2UQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973978998 (397h12m2.744620333s) [noise=true , address=45.91.95.5:4019 , piece_id=WQBPU5OUOFBIT6W6NAUTNFMFI3UU7MBK64U3UET7HN4X7TMCYY3A, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974101712 (397h12m2.744743047s) [noise=true , address=104.36.82.35:4593 , piece_id=CP5TQJXAV2OWWXENMFF44P5DI5QU5HIYWAXH525ITTIR4NDTZHLA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974232022 (397h12m2.744873357s) [noise=true , address=45.91.95.3:4003 , piece_id=OEBEE652D53YZZYWIWIKA7GFVBFIJ2FR5VMNOA7HKNNFWOQBDHVQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974306995 (397h12m2.74494833s) [noise=true , address=104.36.82.3:4774 , piece_id=4EJYB2DWB736CXZ7CTQOE442AZWQJ2TCDTMMCMKXSANWBUIIABLA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974500055 (397h12m2.74514139s) [noise=true , address=77.90.31.21:4894 , piece_id=56U7WC7KQ2S7XLBDNVI7ADMZQX552THVHVK4DANCJF5FJ4XU7TOA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974522848 (397h12m2.745164183s) [noise=true , address=45.91.95.2:4002 , piece_id=N4RIOYFXEIYYJ2DOO3GRRGLPUBNSFDG2OHNXC6TMSYH2KX3TUWXQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974681611 (397h12m2.745322946s) [noise=true , address=104.36.82.18:4369 , piece_id=7EW62VOKMH6CXC4CURNXOAS65EXFSIARSR7MXLT2MAS2SJGQWFHQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974766505 (397h12m2.74540783s) [noise=true , address=77.90.31.19:4859 , piece_id=SZQ4CMWGIVVHARJOMKKTN6DI7VAZGVJQPXZKYCTUPPRVU5ZNGCVQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974923444 (397h12m2.745564779s) [noise=true , address=77.90.31.15:4701 , piece_id=T42LFWTYWBBQFFDYHVU6U5YO2H3RUKIJR2DI3U4FSXY4WEU5ZTFA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.975010481 (397h12m2.745651816s) [noise=true , address=77.90.31.14:4690 , piece_id=XT6BVPU5ZEAGNEPCTPXD7D35GBMUHGD2OGGEY2P22QIQM4EAYELQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.975147763 (397h12m2.745789088s) [noise=true , address=104.36.82.17:4345 , piece_id=QXC4VIW4NJ5376SZZLSLG62CGH4YUSHO77UQPGFVKBHKALFX6XOQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:29.327909891 (397h12m4.098551226s) [upload=317782]: Next returned nil segment
Jul 9 22:22:29.327933787 (397h12m4.098575122s) [upload=317782]: Waiting for error group managing segments...
Jul 9 22:22:29.978258832 (397h12m4.748900167s) [noise=true , address=77.90.31.12:4594 , piece_id=LJJXKPHBZKSNCJTOQFDOYBTL4U7ZBSE7RI57EWMI5CZJ5YD5YI3A, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.979151280 (397h12m4.749792615s) [noise=true , address=77.90.31.21:4894 , piece_id=56U7WC7KQ2S7XLBDNVI7ADMZQX552THVHVK4DANCJF5FJ4XU7TOA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.984195199 (397h12m4.754836534s) [noise=true , address=77.90.31.19:4859 , piece_id=SZQ4CMWGIVVHARJOMKKTN6DI7VAZGVJQPXZKYCTUPPRVU5ZNGCVQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.988046737 (397h12m4.758688072s) [noise=true , address=77.90.31.15:4701 , piece_id=T42LFWTYWBBQFFDYHVU6U5YO2H3RUKIJR2DI3U4FSXY4WEU5ZTFA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.992300966 (397h12m4.762942301s) [noise=true , address=77.90.31.16:4749 , piece_id=IUTCAJL2PVYOLFP33YMWX2B5XP2D2FYVR6UBGKNC7QL6OHCMRL6Q, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.996959930 (397h12m4.767601265s) [noise=true , address=77.90.31.17:4781 , piece_id=LCSMEGJZ35OL7TEOKSTMPROLECXWA6MXWDM3QY3GQZ3LBFA4C2UQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.997648557 (397h12m4.768289892s) [noise=true , address=77.90.31.14:4690 , piece_id=XT6BVPU5ZEAGNEPCTPXD7D35GBMUHGD2OGGEY2P22QIQM4EAYELQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.008095954 (397h12m4.778737289s) [noise=true , address=45.91.116.10:4090 , piece_id=PGHOH3Q7RSNSEOC6QAELKNK6RR5GFURWMOGXEBX7DZ64WXNRD4IA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.008404595 (397h12m4.77904593s) [noise=true , address=77.90.31.13:4649 , piece_id=NYXVP7IV6MQBMUWGECBVUUZ3RYF5XQUXTBMJQFPX3GPA2E4FMTEQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.392350092 (397h12m5.162991417s) [noise=true , address=79.127.220.97:4371 , piece_id=7C6V6Z6P66SYVSA2NSHUKHPVIV22RS4PASY26INJYPF5DGDDABXA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:24.652397067 (397h12m59.423038402s) [noise=true , address=104.36.82.17:4345 , piece_id=QXC4VIW4NJ5376SZZLSLG62CGH4YUSHO77UQPGFVKBHKALFX6XOQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:26.282002097 (397h13m1.052643432s) [noise=true , address=104.36.82.35:4593 , piece_id=CP5TQJXAV2OWWXENMFF44P5DI5QU5HIYWAXH525ITTIR4NDTZHLA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:26.892929169 (397h13m1.663570504s) [noise=true , address=45.91.95.5:4019 , piece_id=WQBPU5OUOFBIT6W6NAUTNFMFI3UU7MBK64U3UET7HN4X7TMCYY3A, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:45.91.95.5:4019): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.892983052 (397h13m1.663624377s) [noise=true , address=104.36.82.19:4429 , piece_id=T6LIMFSHHRXZMQYYXDFTZVZ3GATOMH5LGHCCDLD4MFHK5XU2PSSQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1Bi1qafYFw9NukFHUJzV9M4Jd1i7YsjQMs7jXsVUn44pecVWzr, address:104.36.82.19:4429): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893020182 (397h13m1.663661517s) [noise=true , address=45.91.95.3:4003 , piece_id=OEBEE652D53YZZYWIWIKA7GFVBFIJ2FR5VMNOA7HKNNFWOQBDHVQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:12775mgnBsA3595WztdcNsUBVWCLeeZECij1L2EnwuvzCCtMouk, address:45.91.95.3:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893081459 (397h13m1.663722784s) [noise=true , address=104.36.82.3:4774 , piece_id=4EJYB2DWB736CXZ7CTQOE442AZWQJ2TCDTMMCMKXSANWBUIIABLA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1r3Ty7Q8v3b4nT3cVVzQqGDFpHdBu1XKPQvBy221Y78hPzv4Y9, address:104.36.82.3:4774): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893095246 (397h13m1.663736581s) [noise=true , address=104.36.82.18:4369 , piece_id=7EW62VOKMH6CXC4CURNXOAS65EXFSIARSR7MXLT2MAS2SJGQWFHQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1nwygPHyRJEZkjCoSZJTVhXDKLHc7AonVMzKMht6ppwZwhcGCT, address:104.36.82.18:4369): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893127879 (397h13m1.663769214s) [noise=true , address=45.91.95.2:4002 , piece_id=N4RIOYFXEIYYJ2DOO3GRRGLPUBNSFDG2OHNXC6TMSYH2KX3TUWXQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection; write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection
Jul 9 22:23:26.893140864 (397h13m1.663782199s) [seg_pos={0 0}, upload=317782]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Jul 9 22:23:27.126246135 (397h13m1.89688746s) [upload=317782]: Done uploading segments...
Jul 9 22:23:27.126270812 (397h13m1.896912147s) [upload=317782]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 17.2223468s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.....X.X.............X.........................X............] duration:
6.101301534s
FAIL (2025-07-09 22:45:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.101323064s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.X................X.....................X...................] duration:
22.500220915s
FAIL (2025-07-09 22:45:09 UTC)
orbiter warning: context deadline exceeded
Duration: 22.500280108s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [......X.X............X.........X............................] duration:
22.260854126s
FAIL (2025-07-09 22:45:05 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
:27.530694236 (397h12m2.301335571s) [upload=317782]: Got next segment. Inline: false
Jul 9 22:22:27.530721418 (397h12m2.301362743s) [upload=317782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 22:22:27.530725656 (397h12m2.301366981s) [upload=317782]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 22:22:27.972697777 (397h12m2.743339102s) [seg_pos={0 0}, upload=317782]: Begin upload segment...
Jul 9 22:22:27.972768972 (397h12m2.743410308s) [seg_pos={0 0}, upload=317782]: Done begin upload segment.
Jul 9 22:22:27.972793770 (397h12m2.743435105s) [noise=true , address=77.90.31.13:4649 , piece_id=NYXVP7IV6MQBMUWGECBVUUZ3RYF5XQUXTBMJQFPX3GPA2E4FMTEQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973170050 (397h12m2.743811385s) [noise=true , address=77.90.31.16:4749 , piece_id=IUTCAJL2PVYOLFP33YMWX2B5XP2D2FYVR6UBGKNC7QL6OHCMRL6Q, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973207692 (397h12m2.743849027s) [noise=true , address=104.36.82.19:4429 , piece_id=T6LIMFSHHRXZMQYYXDFTZVZ3GATOMH5LGHCCDLD4MFHK5XU2PSSQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973462760 (397h12m2.744104095s) [noise=true , address=45.91.116.10:4090 , piece_id=PGHOH3Q7RSNSEOC6QAELKNK6RR5GFURWMOGXEBX7DZ64WXNRD4IA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973561820 (397h12m2.744203155s) [noise=true , address=79.127.220.97:4371 , piece_id=7C6V6Z6P66SYVSA2NSHUKHPVIV22RS4PASY26INJYPF5DGDDABXA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973719421 (397h12m2.744360756s) [noise=true , address=77.90.31.12:4594 , piece_id=LJJXKPHBZKSNCJTOQFDOYBTL4U7ZBSE7RI57EWMI5CZJ5YD5YI3A, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973862605 (397h12m2.74450394s) [noise=true , address=77.90.31.17:4781 , piece_id=LCSMEGJZ35OL7TEOKSTMPROLECXWA6MXWDM3QY3GQZ3LBFA4C2UQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.973978998 (397h12m2.744620333s) [noise=true , address=45.91.95.5:4019 , piece_id=WQBPU5OUOFBIT6W6NAUTNFMFI3UU7MBK64U3UET7HN4X7TMCYY3A, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974101712 (397h12m2.744743047s) [noise=true , address=104.36.82.35:4593 , piece_id=CP5TQJXAV2OWWXENMFF44P5DI5QU5HIYWAXH525ITTIR4NDTZHLA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974232022 (397h12m2.744873357s) [noise=true , address=45.91.95.3:4003 , piece_id=OEBEE652D53YZZYWIWIKA7GFVBFIJ2FR5VMNOA7HKNNFWOQBDHVQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974306995 (397h12m2.74494833s) [noise=true , address=104.36.82.3:4774 , piece_id=4EJYB2DWB736CXZ7CTQOE442AZWQJ2TCDTMMCMKXSANWBUIIABLA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974500055 (397h12m2.74514139s) [noise=true , address=77.90.31.21:4894 , piece_id=56U7WC7KQ2S7XLBDNVI7ADMZQX552THVHVK4DANCJF5FJ4XU7TOA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974522848 (397h12m2.745164183s) [noise=true , address=45.91.95.2:4002 , piece_id=N4RIOYFXEIYYJ2DOO3GRRGLPUBNSFDG2OHNXC6TMSYH2KX3TUWXQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974681611 (397h12m2.745322946s) [noise=true , address=104.36.82.18:4369 , piece_id=7EW62VOKMH6CXC4CURNXOAS65EXFSIARSR7MXLT2MAS2SJGQWFHQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974766505 (397h12m2.74540783s) [noise=true , address=77.90.31.19:4859 , piece_id=SZQ4CMWGIVVHARJOMKKTN6DI7VAZGVJQPXZKYCTUPPRVU5ZNGCVQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.974923444 (397h12m2.745564779s) [noise=true , address=77.90.31.15:4701 , piece_id=T42LFWTYWBBQFFDYHVU6U5YO2H3RUKIJR2DI3U4FSXY4WEU5ZTFA, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.975010481 (397h12m2.745651816s) [noise=true , address=77.90.31.14:4690 , piece_id=XT6BVPU5ZEAGNEPCTPXD7D35GBMUHGD2OGGEY2P22QIQM4EAYELQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:27.975147763 (397h12m2.745789088s) [noise=true , address=104.36.82.17:4345 , piece_id=QXC4VIW4NJ5376SZZLSLG62CGH4YUSHO77UQPGFVKBHKALFX6XOQ, seg_pos={0 0}, upload=317782]: Uploading piece...
Jul 9 22:22:29.327909891 (397h12m4.098551226s) [upload=317782]: Next returned nil segment
Jul 9 22:22:29.327933787 (397h12m4.098575122s) [upload=317782]: Waiting for error group managing segments...
Jul 9 22:22:29.978258832 (397h12m4.748900167s) [noise=true , address=77.90.31.12:4594 , piece_id=LJJXKPHBZKSNCJTOQFDOYBTL4U7ZBSE7RI57EWMI5CZJ5YD5YI3A, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.979151280 (397h12m4.749792615s) [noise=true , address=77.90.31.21:4894 , piece_id=56U7WC7KQ2S7XLBDNVI7ADMZQX552THVHVK4DANCJF5FJ4XU7TOA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.984195199 (397h12m4.754836534s) [noise=true , address=77.90.31.19:4859 , piece_id=SZQ4CMWGIVVHARJOMKKTN6DI7VAZGVJQPXZKYCTUPPRVU5ZNGCVQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.988046737 (397h12m4.758688072s) [noise=true , address=77.90.31.15:4701 , piece_id=T42LFWTYWBBQFFDYHVU6U5YO2H3RUKIJR2DI3U4FSXY4WEU5ZTFA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.992300966 (397h12m4.762942301s) [noise=true , address=77.90.31.16:4749 , piece_id=IUTCAJL2PVYOLFP33YMWX2B5XP2D2FYVR6UBGKNC7QL6OHCMRL6Q, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.996959930 (397h12m4.767601265s) [noise=true , address=77.90.31.17:4781 , piece_id=LCSMEGJZ35OL7TEOKSTMPROLECXWA6MXWDM3QY3GQZ3LBFA4C2UQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:29.997648557 (397h12m4.768289892s) [noise=true , address=77.90.31.14:4690 , piece_id=XT6BVPU5ZEAGNEPCTPXD7D35GBMUHGD2OGGEY2P22QIQM4EAYELQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.008095954 (397h12m4.778737289s) [noise=true , address=45.91.116.10:4090 , piece_id=PGHOH3Q7RSNSEOC6QAELKNK6RR5GFURWMOGXEBX7DZ64WXNRD4IA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.008404595 (397h12m4.77904593s) [noise=true , address=77.90.31.13:4649 , piece_id=NYXVP7IV6MQBMUWGECBVUUZ3RYF5XQUXTBMJQFPX3GPA2E4FMTEQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:22:30.392350092 (397h12m5.162991417s) [noise=true , address=79.127.220.97:4371 , piece_id=7C6V6Z6P66SYVSA2NSHUKHPVIV22RS4PASY26INJYPF5DGDDABXA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:24.652397067 (397h12m59.423038402s) [noise=true , address=104.36.82.17:4345 , piece_id=QXC4VIW4NJ5376SZZLSLG62CGH4YUSHO77UQPGFVKBHKALFX6XOQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:26.282002097 (397h13m1.052643432s) [noise=true , address=104.36.82.35:4593 , piece_id=CP5TQJXAV2OWWXENMFF44P5DI5QU5HIYWAXH525ITTIR4NDTZHLA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: <nil>
Jul 9 22:23:26.892929169 (397h13m1.663570504s) [noise=true , address=45.91.95.5:4019 , piece_id=WQBPU5OUOFBIT6W6NAUTNFMFI3UU7MBK64U3UET7HN4X7TMCYY3A, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:45.91.95.5:4019): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.892983052 (397h13m1.663624377s) [noise=true , address=104.36.82.19:4429 , piece_id=T6LIMFSHHRXZMQYYXDFTZVZ3GATOMH5LGHCCDLD4MFHK5XU2PSSQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1Bi1qafYFw9NukFHUJzV9M4Jd1i7YsjQMs7jXsVUn44pecVWzr, address:104.36.82.19:4429): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893020182 (397h13m1.663661517s) [noise=true , address=45.91.95.3:4003 , piece_id=OEBEE652D53YZZYWIWIKA7GFVBFIJ2FR5VMNOA7HKNNFWOQBDHVQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:12775mgnBsA3595WztdcNsUBVWCLeeZECij1L2EnwuvzCCtMouk, address:45.91.95.3:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893081459 (397h13m1.663722784s) [noise=true , address=104.36.82.3:4774 , piece_id=4EJYB2DWB736CXZ7CTQOE442AZWQJ2TCDTMMCMKXSANWBUIIABLA, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1r3Ty7Q8v3b4nT3cVVzQqGDFpHdBu1XKPQvBy221Y78hPzv4Y9, address:104.36.82.3:4774): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893095246 (397h13m1.663736581s) [noise=true , address=104.36.82.18:4369 , piece_id=7EW62VOKMH6CXC4CURNXOAS65EXFSIARSR7MXLT2MAS2SJGQWFHQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1nwygPHyRJEZkjCoSZJTVhXDKLHc7AonVMzKMht6ppwZwhcGCT, address:104.36.82.18:4369): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 22:23:26.893127879 (397h13m1.663769214s) [noise=true , address=45.91.95.2:4002 , piece_id=N4RIOYFXEIYYJ2DOO3GRRGLPUBNSFDG2OHNXC6TMSYH2KX3TUWXQ, seg_pos={0 0}, upload=317782]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection; write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39010->45.91.95.2:4002: use of closed network connection
Jul 9 22:23:26.893140864 (397h13m1.663782199s) [seg_pos={0 0}, upload=317782]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Jul 9 22:23:27.126246135 (397h13m1.89688746s) [upload=317782]: Done uploading segments...
Jul 9 22:23:27.126270812 (397h13m1.896912147s) [upload=317782]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 22.260889974s
core
uplink
us-select-1
us1-placement-20
|