Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.134892987s
FAIL (2025-08-06 17:04:21 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 2.134924568s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.........X.....] duration:
602.236782ms remote:
"tcp:3.109.243.18:443" response headers:
Date=Wed, 06 Aug 2025 17:01:25 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-06 17:04:00 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 602.260026ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [X..........................X.........X....X.................] duration:
5.625613401s
FAIL (2025-08-06 17:03:52 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.625646424s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.232419522s
FAIL (2025-08-06 17:03:43 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 2.232443017s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [..........X....] duration:
616.401294ms remote:
"tcp:3.109.243.18:443" response headers:
Date=Wed, 06 Aug 2025 17:01:25 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-06 17:03:40 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 616.424046ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [...........X...] duration:
797.1409ms remote:
"tcp:3.109.243.18:443" response headers:
Date=Wed, 06 Aug 2025 17:01:25 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-06 17:03:08 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 797.157483ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.X..........................X.........X....X................] duration:
6.087964029s
FAIL (2025-08-06 17:03:04 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 6.087989478s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.226585259s
FAIL (2025-08-06 17:02:48 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 2.226612843s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.0864555s
FAIL (2025-08-06 17:02:11 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 2.086494916s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [..X..........................X.........X....X...............] duration:
6.157389983s
FAIL (2025-08-06 17:02:10 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 6.157418727s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [............X..] duration:
638.03767ms remote:
"tcp:3.109.243.18:443" response headers:
Date=Wed, 06 Aug 2025 17:01:25 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-06 17:02:09 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 638.061836ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.............X.] duration:
633.873333ms remote:
"tcp:3.109.243.18:443" response headers:
Date=Wed, 06 Aug 2025 17:01:25 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-06 17:01:50 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 633.893741ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.986732022s
FAIL (2025-08-06 17:01:39 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.986755687s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [X..X..........................X.........X....X..............] duration:
6.134566954s
FAIL (2025-08-06 17:01:32 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 6.134593955s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [..............X] duration:
20.252429133s remote:
"tcp:3.109.243.18:443" response headers:
Date=Wed, 06 Aug 2025 17:01:25 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-06 17:01:25 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 20.252449691s
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.X..X..........................X.........X....X.............] duration:
5.85026845s
FAIL (2025-08-06 17:00:37 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.850296894s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.865538763s
FAIL (2025-08-06 17:00:36 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.865565343s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.839651635s
FAIL (2025-08-06 16:59:32 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.839677003s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [X.X..X..........................X.........X....X............] duration:
6.678002262s
FAIL (2025-08-06 16:59:28 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 6.678048853s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.156992458s
FAIL (2025-08-06 16:58:58 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 2.157017777s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.X.X..X..........................X.........X....X...........] duration:
5.975865773s
FAIL (2025-08-06 16:58:29 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.975886903s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [..X.X..X..........................X.........X....X..........] duration:
5.742815782s
FAIL (2025-08-06 16:57:50 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.742839807s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.824409164s
FAIL (2025-08-06 16:57:45 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
cted piece hash; context canceled; EOF
Aug 6 16:15:07.596899502 (2h6m15.535315351s) [noise=true , address=104.36.82.12:4151 , piece_id=B4D7VTLBQBRSPKA7N42KZTE7V5TSS754ZV7EF75JJEFDOW55O4CQ, seg_pos={0 0}, upload=2001]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Aug 6 16:15:07.596915132 (2h6m15.535330981s) [seg_pos={0 0}, upload=2001]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:15:07.596991619 (2h6m15.535407468s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:15:07.596995857 (2h6m15.535411706s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:15:07.887184563 (2h6m15.825600412s) [upload=2001]: Done uploading segments...
Aug 6 16:15:07.887203770 (2h6m15.825619609s) [upload=2001]: Upload finished. err: <nil>
Aug 6 16:16:00.579639900 (2h7m8.518055749s) [upload=2016]: Starting upload
Aug 6 16:16:00.579694645 (2h7m8.518110494s) [upload=2016]: Done starting upload
Aug 6 16:16:00.579789666 (2h7m8.518205515s) [upload=2016]: Uploading segments...
Aug 6 16:16:00.579861003 (2h7m8.518276852s) [upload=2016]: Got next segment. Inline: false
Aug 6 16:16:00.579885871 (2h7m8.51830172s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 6 16:16:00.579889858 (2h7m8.518305707s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 6 16:16:00.820626959 (2h7m8.759042808s) [seg_pos={0 0}, upload=2016]: Begin upload segment...
Aug 6 16:16:00.820694769 (2h7m8.759110629s) [seg_pos={0 0}, upload=2016]: Done begin upload segment.
Aug 6 16:16:00.820738192 (2h7m8.759154041s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.820754293 (2h7m8.759170142s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821224302 (2h7m8.759640151s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821272084 (2h7m8.759687983s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821617053 (2h7m8.760032903s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822107232 (2h7m8.760523091s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822131258 (2h7m8.760547107s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822500845 (2h7m8.760916694s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822543386 (2h7m8.760959235s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822973009 (2h7m8.761388858s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823204822 (2h7m8.761620661s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823426235 (2h7m8.761842084s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823693006 (2h7m8.762108855s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823992970 (2h7m8.762408819s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824130774 (2h7m8.762546623s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824422621 (2h7m8.76283848s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:02.555639143 (2h7m10.494054992s) [upload=2016]: Next returned nil segment
Aug 6 16:16:02.555659893 (2h7m10.494075752s) [upload=2016]: Waiting for error group managing segments...
Aug 6 16:16:02.837308740 (2h7m10.775724589s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.882019636 (2h7m10.820435485s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.917400741 (2h7m10.85581659s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.947806114 (2h7m10.886221963s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.964255246 (2h7m10.902671095s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972488429 (2h7m10.910904278s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972812389 (2h7m10.911228238s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.001575382 (2h7m10.939991231s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.028168973 (2h7m10.966584822s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.036656182 (2h7m10.975072031s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.044792840 (2h7m10.983208689s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.045187775 (2h7m10.983603634s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118252640 (2h7m11.056668479s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118270995 (2h7m11.056686844s) [seg_pos={0 0}, upload=2016]: Segment reached optimal threshold of 13 pieces.
Aug 6 16:16:03.118456620 (2h7m11.056872469s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119323248 (2h7m11.057739097s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119734826 (2h7m11.058150675s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection
Aug 6 16:16:03.119748902 (2h7m11.058164751s) [seg_pos={0 0}, upload=2016]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:16:03.119818285 (2h7m11.058234134s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:16:03.119823415 (2h7m11.058239264s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:16:03.416354947 (2h7m11.354770796s) [upload=2016]: Done uploading segments...
Aug 6 16:16:03.416374936 (2h7m11.354790785s) [upload=2016]: Upload finished. err: <nil>
Duration: 5.824447547s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.91545662s
FAIL (2025-08-06 16:57:45 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.91549768s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.050058141s
FAIL (2025-08-06 16:57:22 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 2.05010918s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.091046865s
FAIL (2025-08-06 16:56:48 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 2.091073055s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
5.980356095s
FAIL (2025-08-06 16:56:35 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
cted piece hash; context canceled; EOF
Aug 6 16:15:07.596899502 (2h6m15.535315351s) [noise=true , address=104.36.82.12:4151 , piece_id=B4D7VTLBQBRSPKA7N42KZTE7V5TSS754ZV7EF75JJEFDOW55O4CQ, seg_pos={0 0}, upload=2001]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Aug 6 16:15:07.596915132 (2h6m15.535330981s) [seg_pos={0 0}, upload=2001]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:15:07.596991619 (2h6m15.535407468s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:15:07.596995857 (2h6m15.535411706s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:15:07.887184563 (2h6m15.825600412s) [upload=2001]: Done uploading segments...
Aug 6 16:15:07.887203770 (2h6m15.825619609s) [upload=2001]: Upload finished. err: <nil>
Aug 6 16:16:00.579639900 (2h7m8.518055749s) [upload=2016]: Starting upload
Aug 6 16:16:00.579694645 (2h7m8.518110494s) [upload=2016]: Done starting upload
Aug 6 16:16:00.579789666 (2h7m8.518205515s) [upload=2016]: Uploading segments...
Aug 6 16:16:00.579861003 (2h7m8.518276852s) [upload=2016]: Got next segment. Inline: false
Aug 6 16:16:00.579885871 (2h7m8.51830172s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 6 16:16:00.579889858 (2h7m8.518305707s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 6 16:16:00.820626959 (2h7m8.759042808s) [seg_pos={0 0}, upload=2016]: Begin upload segment...
Aug 6 16:16:00.820694769 (2h7m8.759110629s) [seg_pos={0 0}, upload=2016]: Done begin upload segment.
Aug 6 16:16:00.820738192 (2h7m8.759154041s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.820754293 (2h7m8.759170142s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821224302 (2h7m8.759640151s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821272084 (2h7m8.759687983s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821617053 (2h7m8.760032903s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822107232 (2h7m8.760523091s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822131258 (2h7m8.760547107s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822500845 (2h7m8.760916694s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822543386 (2h7m8.760959235s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822973009 (2h7m8.761388858s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823204822 (2h7m8.761620661s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823426235 (2h7m8.761842084s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823693006 (2h7m8.762108855s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823992970 (2h7m8.762408819s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824130774 (2h7m8.762546623s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824422621 (2h7m8.76283848s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:02.555639143 (2h7m10.494054992s) [upload=2016]: Next returned nil segment
Aug 6 16:16:02.555659893 (2h7m10.494075752s) [upload=2016]: Waiting for error group managing segments...
Aug 6 16:16:02.837308740 (2h7m10.775724589s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.882019636 (2h7m10.820435485s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.917400741 (2h7m10.85581659s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.947806114 (2h7m10.886221963s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.964255246 (2h7m10.902671095s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972488429 (2h7m10.910904278s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972812389 (2h7m10.911228238s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.001575382 (2h7m10.939991231s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.028168973 (2h7m10.966584822s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.036656182 (2h7m10.975072031s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.044792840 (2h7m10.983208689s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.045187775 (2h7m10.983603634s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118252640 (2h7m11.056668479s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118270995 (2h7m11.056686844s) [seg_pos={0 0}, upload=2016]: Segment reached optimal threshold of 13 pieces.
Aug 6 16:16:03.118456620 (2h7m11.056872469s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119323248 (2h7m11.057739097s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119734826 (2h7m11.058150675s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection
Aug 6 16:16:03.119748902 (2h7m11.058164751s) [seg_pos={0 0}, upload=2016]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:16:03.119818285 (2h7m11.058234134s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:16:03.119823415 (2h7m11.058239264s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:16:03.416354947 (2h7m11.354770796s) [upload=2016]: Done uploading segments...
Aug 6 16:16:03.416374936 (2h7m11.354790785s) [upload=2016]: Upload finished. err: <nil>
Duration: 5.980391121s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [...X.X..X..........................X.........X....X.........] duration:
5.722380524s
FAIL (2025-08-06 16:56:35 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.722404358s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.848235898s
FAIL (2025-08-06 16:56:11 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.848261738s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
5.786234866s
FAIL (2025-08-06 16:55:40 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
cted piece hash; context canceled; EOF
Aug 6 16:15:07.596899502 (2h6m15.535315351s) [noise=true , address=104.36.82.12:4151 , piece_id=B4D7VTLBQBRSPKA7N42KZTE7V5TSS754ZV7EF75JJEFDOW55O4CQ, seg_pos={0 0}, upload=2001]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Aug 6 16:15:07.596915132 (2h6m15.535330981s) [seg_pos={0 0}, upload=2001]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:15:07.596991619 (2h6m15.535407468s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:15:07.596995857 (2h6m15.535411706s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:15:07.887184563 (2h6m15.825600412s) [upload=2001]: Done uploading segments...
Aug 6 16:15:07.887203770 (2h6m15.825619609s) [upload=2001]: Upload finished. err: <nil>
Aug 6 16:16:00.579639900 (2h7m8.518055749s) [upload=2016]: Starting upload
Aug 6 16:16:00.579694645 (2h7m8.518110494s) [upload=2016]: Done starting upload
Aug 6 16:16:00.579789666 (2h7m8.518205515s) [upload=2016]: Uploading segments...
Aug 6 16:16:00.579861003 (2h7m8.518276852s) [upload=2016]: Got next segment. Inline: false
Aug 6 16:16:00.579885871 (2h7m8.51830172s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 6 16:16:00.579889858 (2h7m8.518305707s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 6 16:16:00.820626959 (2h7m8.759042808s) [seg_pos={0 0}, upload=2016]: Begin upload segment...
Aug 6 16:16:00.820694769 (2h7m8.759110629s) [seg_pos={0 0}, upload=2016]: Done begin upload segment.
Aug 6 16:16:00.820738192 (2h7m8.759154041s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.820754293 (2h7m8.759170142s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821224302 (2h7m8.759640151s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821272084 (2h7m8.759687983s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821617053 (2h7m8.760032903s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822107232 (2h7m8.760523091s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822131258 (2h7m8.760547107s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822500845 (2h7m8.760916694s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822543386 (2h7m8.760959235s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822973009 (2h7m8.761388858s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823204822 (2h7m8.761620661s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823426235 (2h7m8.761842084s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823693006 (2h7m8.762108855s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823992970 (2h7m8.762408819s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824130774 (2h7m8.762546623s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824422621 (2h7m8.76283848s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:02.555639143 (2h7m10.494054992s) [upload=2016]: Next returned nil segment
Aug 6 16:16:02.555659893 (2h7m10.494075752s) [upload=2016]: Waiting for error group managing segments...
Aug 6 16:16:02.837308740 (2h7m10.775724589s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.882019636 (2h7m10.820435485s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.917400741 (2h7m10.85581659s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.947806114 (2h7m10.886221963s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.964255246 (2h7m10.902671095s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972488429 (2h7m10.910904278s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972812389 (2h7m10.911228238s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.001575382 (2h7m10.939991231s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.028168973 (2h7m10.966584822s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.036656182 (2h7m10.975072031s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.044792840 (2h7m10.983208689s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.045187775 (2h7m10.983603634s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118252640 (2h7m11.056668479s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118270995 (2h7m11.056686844s) [seg_pos={0 0}, upload=2016]: Segment reached optimal threshold of 13 pieces.
Aug 6 16:16:03.118456620 (2h7m11.056872469s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119323248 (2h7m11.057739097s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119734826 (2h7m11.058150675s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection
Aug 6 16:16:03.119748902 (2h7m11.058164751s) [seg_pos={0 0}, upload=2016]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:16:03.119818285 (2h7m11.058234134s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:16:03.119823415 (2h7m11.058239264s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:16:03.416354947 (2h7m11.354770796s) [upload=2016]: Done uploading segments...
Aug 6 16:16:03.416374936 (2h7m11.354790785s) [upload=2016]: Upload finished. err: <nil>
Duration: 5.786256388s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [....X.X..X..........................X.........X....X........] duration:
5.852039186s
FAIL (2025-08-06 16:55:25 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.852054095s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.9266258s
FAIL (2025-08-06 16:55:23 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.9266519s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.....X.X..X..........................X.........X....X.......] duration:
5.6709753s
FAIL (2025-08-06 16:55:08 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.670999956s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.223792442s
FAIL (2025-08-06 16:55:03 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 2.223851084s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
5.776512621s
FAIL (2025-08-06 16:54:46 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
cted piece hash; context canceled; EOF
Aug 6 16:15:07.596899502 (2h6m15.535315351s) [noise=true , address=104.36.82.12:4151 , piece_id=B4D7VTLBQBRSPKA7N42KZTE7V5TSS754ZV7EF75JJEFDOW55O4CQ, seg_pos={0 0}, upload=2001]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Aug 6 16:15:07.596915132 (2h6m15.535330981s) [seg_pos={0 0}, upload=2001]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:15:07.596991619 (2h6m15.535407468s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:15:07.596995857 (2h6m15.535411706s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:15:07.887184563 (2h6m15.825600412s) [upload=2001]: Done uploading segments...
Aug 6 16:15:07.887203770 (2h6m15.825619609s) [upload=2001]: Upload finished. err: <nil>
Aug 6 16:16:00.579639900 (2h7m8.518055749s) [upload=2016]: Starting upload
Aug 6 16:16:00.579694645 (2h7m8.518110494s) [upload=2016]: Done starting upload
Aug 6 16:16:00.579789666 (2h7m8.518205515s) [upload=2016]: Uploading segments...
Aug 6 16:16:00.579861003 (2h7m8.518276852s) [upload=2016]: Got next segment. Inline: false
Aug 6 16:16:00.579885871 (2h7m8.51830172s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 6 16:16:00.579889858 (2h7m8.518305707s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 6 16:16:00.820626959 (2h7m8.759042808s) [seg_pos={0 0}, upload=2016]: Begin upload segment...
Aug 6 16:16:00.820694769 (2h7m8.759110629s) [seg_pos={0 0}, upload=2016]: Done begin upload segment.
Aug 6 16:16:00.820738192 (2h7m8.759154041s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.820754293 (2h7m8.759170142s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821224302 (2h7m8.759640151s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821272084 (2h7m8.759687983s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821617053 (2h7m8.760032903s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822107232 (2h7m8.760523091s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822131258 (2h7m8.760547107s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822500845 (2h7m8.760916694s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822543386 (2h7m8.760959235s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822973009 (2h7m8.761388858s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823204822 (2h7m8.761620661s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823426235 (2h7m8.761842084s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823693006 (2h7m8.762108855s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823992970 (2h7m8.762408819s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824130774 (2h7m8.762546623s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824422621 (2h7m8.76283848s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:02.555639143 (2h7m10.494054992s) [upload=2016]: Next returned nil segment
Aug 6 16:16:02.555659893 (2h7m10.494075752s) [upload=2016]: Waiting for error group managing segments...
Aug 6 16:16:02.837308740 (2h7m10.775724589s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.882019636 (2h7m10.820435485s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.917400741 (2h7m10.85581659s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.947806114 (2h7m10.886221963s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.964255246 (2h7m10.902671095s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972488429 (2h7m10.910904278s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972812389 (2h7m10.911228238s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.001575382 (2h7m10.939991231s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.028168973 (2h7m10.966584822s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.036656182 (2h7m10.975072031s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.044792840 (2h7m10.983208689s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.045187775 (2h7m10.983603634s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118252640 (2h7m11.056668479s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118270995 (2h7m11.056686844s) [seg_pos={0 0}, upload=2016]: Segment reached optimal threshold of 13 pieces.
Aug 6 16:16:03.118456620 (2h7m11.056872469s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119323248 (2h7m11.057739097s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119734826 (2h7m11.058150675s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection
Aug 6 16:16:03.119748902 (2h7m11.058164751s) [seg_pos={0 0}, upload=2016]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:16:03.119818285 (2h7m11.058234134s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:16:03.119823415 (2h7m11.058239264s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:16:03.416354947 (2h7m11.354770796s) [upload=2016]: Done uploading segments...
Aug 6 16:16:03.416374936 (2h7m11.354790785s) [upload=2016]: Upload finished. err: <nil>
Duration: 5.776540484s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.928644372s
FAIL (2025-08-06 16:54:42 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.928672507s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [......X.X..X..........................X.........X....X......] duration:
5.611451133s
FAIL (2025-08-06 16:54:31 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.61147065s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
5.974063515s
FAIL (2025-08-06 16:53:51 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
cted piece hash; context canceled; EOF
Aug 6 16:15:07.596899502 (2h6m15.535315351s) [noise=true , address=104.36.82.12:4151 , piece_id=B4D7VTLBQBRSPKA7N42KZTE7V5TSS754ZV7EF75JJEFDOW55O4CQ, seg_pos={0 0}, upload=2001]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Aug 6 16:15:07.596915132 (2h6m15.535330981s) [seg_pos={0 0}, upload=2001]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:15:07.596991619 (2h6m15.535407468s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:15:07.596995857 (2h6m15.535411706s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:15:07.887184563 (2h6m15.825600412s) [upload=2001]: Done uploading segments...
Aug 6 16:15:07.887203770 (2h6m15.825619609s) [upload=2001]: Upload finished. err: <nil>
Aug 6 16:16:00.579639900 (2h7m8.518055749s) [upload=2016]: Starting upload
Aug 6 16:16:00.579694645 (2h7m8.518110494s) [upload=2016]: Done starting upload
Aug 6 16:16:00.579789666 (2h7m8.518205515s) [upload=2016]: Uploading segments...
Aug 6 16:16:00.579861003 (2h7m8.518276852s) [upload=2016]: Got next segment. Inline: false
Aug 6 16:16:00.579885871 (2h7m8.51830172s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 6 16:16:00.579889858 (2h7m8.518305707s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 6 16:16:00.820626959 (2h7m8.759042808s) [seg_pos={0 0}, upload=2016]: Begin upload segment...
Aug 6 16:16:00.820694769 (2h7m8.759110629s) [seg_pos={0 0}, upload=2016]: Done begin upload segment.
Aug 6 16:16:00.820738192 (2h7m8.759154041s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.820754293 (2h7m8.759170142s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821224302 (2h7m8.759640151s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821272084 (2h7m8.759687983s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821617053 (2h7m8.760032903s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822107232 (2h7m8.760523091s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822131258 (2h7m8.760547107s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822500845 (2h7m8.760916694s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822543386 (2h7m8.760959235s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822973009 (2h7m8.761388858s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823204822 (2h7m8.761620661s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823426235 (2h7m8.761842084s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823693006 (2h7m8.762108855s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823992970 (2h7m8.762408819s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824130774 (2h7m8.762546623s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824422621 (2h7m8.76283848s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:02.555639143 (2h7m10.494054992s) [upload=2016]: Next returned nil segment
Aug 6 16:16:02.555659893 (2h7m10.494075752s) [upload=2016]: Waiting for error group managing segments...
Aug 6 16:16:02.837308740 (2h7m10.775724589s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.882019636 (2h7m10.820435485s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.917400741 (2h7m10.85581659s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.947806114 (2h7m10.886221963s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.964255246 (2h7m10.902671095s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972488429 (2h7m10.910904278s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972812389 (2h7m10.911228238s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.001575382 (2h7m10.939991231s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.028168973 (2h7m10.966584822s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.036656182 (2h7m10.975072031s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.044792840 (2h7m10.983208689s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.045187775 (2h7m10.983603634s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118252640 (2h7m11.056668479s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118270995 (2h7m11.056686844s) [seg_pos={0 0}, upload=2016]: Segment reached optimal threshold of 13 pieces.
Aug 6 16:16:03.118456620 (2h7m11.056872469s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119323248 (2h7m11.057739097s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119734826 (2h7m11.058150675s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection
Aug 6 16:16:03.119748902 (2h7m11.058164751s) [seg_pos={0 0}, upload=2016]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:16:03.119818285 (2h7m11.058234134s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:16:03.119823415 (2h7m11.058239264s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:16:03.416354947 (2h7m11.354770796s) [upload=2016]: Done uploading segments...
Aug 6 16:16:03.416374936 (2h7m11.354790785s) [upload=2016]: Upload finished. err: <nil>
Duration: 5.974118729s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.913048624s
FAIL (2025-08-06 16:53:39 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.913097527s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
5.771978268s
FAIL (2025-08-06 16:53:28 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
cted piece hash; context canceled; EOF
Aug 6 16:15:07.596899502 (2h6m15.535315351s) [noise=true , address=104.36.82.12:4151 , piece_id=B4D7VTLBQBRSPKA7N42KZTE7V5TSS754ZV7EF75JJEFDOW55O4CQ, seg_pos={0 0}, upload=2001]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Aug 6 16:15:07.596915132 (2h6m15.535330981s) [seg_pos={0 0}, upload=2001]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:15:07.596991619 (2h6m15.535407468s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:15:07.596995857 (2h6m15.535411706s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:15:07.887184563 (2h6m15.825600412s) [upload=2001]: Done uploading segments...
Aug 6 16:15:07.887203770 (2h6m15.825619609s) [upload=2001]: Upload finished. err: <nil>
Aug 6 16:16:00.579639900 (2h7m8.518055749s) [upload=2016]: Starting upload
Aug 6 16:16:00.579694645 (2h7m8.518110494s) [upload=2016]: Done starting upload
Aug 6 16:16:00.579789666 (2h7m8.518205515s) [upload=2016]: Uploading segments...
Aug 6 16:16:00.579861003 (2h7m8.518276852s) [upload=2016]: Got next segment. Inline: false
Aug 6 16:16:00.579885871 (2h7m8.51830172s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 6 16:16:00.579889858 (2h7m8.518305707s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 6 16:16:00.820626959 (2h7m8.759042808s) [seg_pos={0 0}, upload=2016]: Begin upload segment...
Aug 6 16:16:00.820694769 (2h7m8.759110629s) [seg_pos={0 0}, upload=2016]: Done begin upload segment.
Aug 6 16:16:00.820738192 (2h7m8.759154041s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.820754293 (2h7m8.759170142s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821224302 (2h7m8.759640151s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821272084 (2h7m8.759687983s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821617053 (2h7m8.760032903s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822107232 (2h7m8.760523091s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822131258 (2h7m8.760547107s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822500845 (2h7m8.760916694s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822543386 (2h7m8.760959235s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822973009 (2h7m8.761388858s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823204822 (2h7m8.761620661s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823426235 (2h7m8.761842084s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823693006 (2h7m8.762108855s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823992970 (2h7m8.762408819s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824130774 (2h7m8.762546623s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824422621 (2h7m8.76283848s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:02.555639143 (2h7m10.494054992s) [upload=2016]: Next returned nil segment
Aug 6 16:16:02.555659893 (2h7m10.494075752s) [upload=2016]: Waiting for error group managing segments...
Aug 6 16:16:02.837308740 (2h7m10.775724589s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.882019636 (2h7m10.820435485s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.917400741 (2h7m10.85581659s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.947806114 (2h7m10.886221963s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.964255246 (2h7m10.902671095s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972488429 (2h7m10.910904278s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972812389 (2h7m10.911228238s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.001575382 (2h7m10.939991231s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.028168973 (2h7m10.966584822s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.036656182 (2h7m10.975072031s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.044792840 (2h7m10.983208689s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.045187775 (2h7m10.983603634s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118252640 (2h7m11.056668479s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118270995 (2h7m11.056686844s) [seg_pos={0 0}, upload=2016]: Segment reached optimal threshold of 13 pieces.
Aug 6 16:16:03.118456620 (2h7m11.056872469s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119323248 (2h7m11.057739097s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119734826 (2h7m11.058150675s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection
Aug 6 16:16:03.119748902 (2h7m11.058164751s) [seg_pos={0 0}, upload=2016]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:16:03.119818285 (2h7m11.058234134s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:16:03.119823415 (2h7m11.058239264s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:16:03.416354947 (2h7m11.354770796s) [upload=2016]: Done uploading segments...
Aug 6 16:16:03.416374936 (2h7m11.354790785s) [upload=2016]: Upload finished. err: <nil>
Duration: 5.772007082s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.......X.X..X..........................X.........X....X.....] duration:
5.812116023s
FAIL (2025-08-06 16:53:26 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.812138166s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
1.879081886s
FAIL (2025-08-06 16:53:19 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.879104629s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.885269744s
FAIL (2025-08-06 16:52:57 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.885292628s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
7 out of 60 is failed (threshold=3) [X.......X.X..X..........................X.........X....X....] duration:
5.734129281s
FAIL (2025-08-06 16:52:46 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.734153999s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
5.848830879s
FAIL (2025-08-06 16:52:42 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
cted piece hash; context canceled; EOF
Aug 6 16:15:07.596899502 (2h6m15.535315351s) [noise=true , address=104.36.82.12:4151 , piece_id=B4D7VTLBQBRSPKA7N42KZTE7V5TSS754ZV7EF75JJEFDOW55O4CQ, seg_pos={0 0}, upload=2001]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Aug 6 16:15:07.596915132 (2h6m15.535330981s) [seg_pos={0 0}, upload=2001]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:15:07.596991619 (2h6m15.535407468s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:15:07.596995857 (2h6m15.535411706s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:15:07.887184563 (2h6m15.825600412s) [upload=2001]: Done uploading segments...
Aug 6 16:15:07.887203770 (2h6m15.825619609s) [upload=2001]: Upload finished. err: <nil>
Aug 6 16:16:00.579639900 (2h7m8.518055749s) [upload=2016]: Starting upload
Aug 6 16:16:00.579694645 (2h7m8.518110494s) [upload=2016]: Done starting upload
Aug 6 16:16:00.579789666 (2h7m8.518205515s) [upload=2016]: Uploading segments...
Aug 6 16:16:00.579861003 (2h7m8.518276852s) [upload=2016]: Got next segment. Inline: false
Aug 6 16:16:00.579885871 (2h7m8.51830172s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 6 16:16:00.579889858 (2h7m8.518305707s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 6 16:16:00.820626959 (2h7m8.759042808s) [seg_pos={0 0}, upload=2016]: Begin upload segment...
Aug 6 16:16:00.820694769 (2h7m8.759110629s) [seg_pos={0 0}, upload=2016]: Done begin upload segment.
Aug 6 16:16:00.820738192 (2h7m8.759154041s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.820754293 (2h7m8.759170142s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821224302 (2h7m8.759640151s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821272084 (2h7m8.759687983s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821617053 (2h7m8.760032903s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822107232 (2h7m8.760523091s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822131258 (2h7m8.760547107s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822500845 (2h7m8.760916694s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822543386 (2h7m8.760959235s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822973009 (2h7m8.761388858s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823204822 (2h7m8.761620661s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823426235 (2h7m8.761842084s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823693006 (2h7m8.762108855s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823992970 (2h7m8.762408819s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824130774 (2h7m8.762546623s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824422621 (2h7m8.76283848s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:02.555639143 (2h7m10.494054992s) [upload=2016]: Next returned nil segment
Aug 6 16:16:02.555659893 (2h7m10.494075752s) [upload=2016]: Waiting for error group managing segments...
Aug 6 16:16:02.837308740 (2h7m10.775724589s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.882019636 (2h7m10.820435485s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.917400741 (2h7m10.85581659s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.947806114 (2h7m10.886221963s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.964255246 (2h7m10.902671095s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972488429 (2h7m10.910904278s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972812389 (2h7m10.911228238s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.001575382 (2h7m10.939991231s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.028168973 (2h7m10.966584822s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.036656182 (2h7m10.975072031s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.044792840 (2h7m10.983208689s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.045187775 (2h7m10.983603634s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118252640 (2h7m11.056668479s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118270995 (2h7m11.056686844s) [seg_pos={0 0}, upload=2016]: Segment reached optimal threshold of 13 pieces.
Aug 6 16:16:03.118456620 (2h7m11.056872469s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119323248 (2h7m11.057739097s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119734826 (2h7m11.058150675s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection
Aug 6 16:16:03.119748902 (2h7m11.058164751s) [seg_pos={0 0}, upload=2016]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:16:03.119818285 (2h7m11.058234134s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:16:03.119823415 (2h7m11.058239264s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:16:03.416354947 (2h7m11.354770796s) [upload=2016]: Done uploading segments...
Aug 6 16:16:03.416374936 (2h7m11.354790785s) [upload=2016]: Upload finished. err: <nil>
Duration: 5.848854245s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.925661867s
FAIL (2025-08-06 16:52:13 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.925685802s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
7 out of 60 is failed (threshold=3) [.X.......X.X..X..........................X.........X....X...] duration:
5.913324984s
FAIL (2025-08-06 16:52:07 UTC)
orbiter warning: operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
Duration: 5.913351496s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
5.801934213s
FAIL (2025-08-06 16:52:06 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
cted piece hash; context canceled; EOF
Aug 6 16:15:07.596899502 (2h6m15.535315351s) [noise=true , address=104.36.82.12:4151 , piece_id=B4D7VTLBQBRSPKA7N42KZTE7V5TSS754ZV7EF75JJEFDOW55O4CQ, seg_pos={0 0}, upload=2001]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Aug 6 16:15:07.596915132 (2h6m15.535330981s) [seg_pos={0 0}, upload=2001]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:15:07.596991619 (2h6m15.535407468s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:15:07.596995857 (2h6m15.535411706s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:15:07.887184563 (2h6m15.825600412s) [upload=2001]: Done uploading segments...
Aug 6 16:15:07.887203770 (2h6m15.825619609s) [upload=2001]: Upload finished. err: <nil>
Aug 6 16:16:00.579639900 (2h7m8.518055749s) [upload=2016]: Starting upload
Aug 6 16:16:00.579694645 (2h7m8.518110494s) [upload=2016]: Done starting upload
Aug 6 16:16:00.579789666 (2h7m8.518205515s) [upload=2016]: Uploading segments...
Aug 6 16:16:00.579861003 (2h7m8.518276852s) [upload=2016]: Got next segment. Inline: false
Aug 6 16:16:00.579885871 (2h7m8.51830172s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 6 16:16:00.579889858 (2h7m8.518305707s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 6 16:16:00.820626959 (2h7m8.759042808s) [seg_pos={0 0}, upload=2016]: Begin upload segment...
Aug 6 16:16:00.820694769 (2h7m8.759110629s) [seg_pos={0 0}, upload=2016]: Done begin upload segment.
Aug 6 16:16:00.820738192 (2h7m8.759154041s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.820754293 (2h7m8.759170142s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821224302 (2h7m8.759640151s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821272084 (2h7m8.759687983s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821617053 (2h7m8.760032903s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822107232 (2h7m8.760523091s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822131258 (2h7m8.760547107s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822500845 (2h7m8.760916694s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822543386 (2h7m8.760959235s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822973009 (2h7m8.761388858s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823204822 (2h7m8.761620661s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823426235 (2h7m8.761842084s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823693006 (2h7m8.762108855s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823992970 (2h7m8.762408819s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824130774 (2h7m8.762546623s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824422621 (2h7m8.76283848s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:02.555639143 (2h7m10.494054992s) [upload=2016]: Next returned nil segment
Aug 6 16:16:02.555659893 (2h7m10.494075752s) [upload=2016]: Waiting for error group managing segments...
Aug 6 16:16:02.837308740 (2h7m10.775724589s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.882019636 (2h7m10.820435485s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.917400741 (2h7m10.85581659s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.947806114 (2h7m10.886221963s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.964255246 (2h7m10.902671095s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972488429 (2h7m10.910904278s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972812389 (2h7m10.911228238s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.001575382 (2h7m10.939991231s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.028168973 (2h7m10.966584822s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.036656182 (2h7m10.975072031s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.044792840 (2h7m10.983208689s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.045187775 (2h7m10.983603634s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118252640 (2h7m11.056668479s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118270995 (2h7m11.056686844s) [seg_pos={0 0}, upload=2016]: Segment reached optimal threshold of 13 pieces.
Aug 6 16:16:03.118456620 (2h7m11.056872469s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119323248 (2h7m11.057739097s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119734826 (2h7m11.058150675s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection
Aug 6 16:16:03.119748902 (2h7m11.058164751s) [seg_pos={0 0}, upload=2016]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:16:03.119818285 (2h7m11.058234134s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:16:03.119823415 (2h7m11.058239264s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:16:03.416354947 (2h7m11.354770796s) [upload=2016]: Done uploading segments...
Aug 6 16:16:03.416374936 (2h7m11.354790785s) [upload=2016]: Upload finished. err: <nil>
Duration: 5.80195929s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
1.834243837s
FAIL (2025-08-06 16:51:51 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("orbiter_us_0/1K")
Duration: 1.834268073s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
5.864971s
FAIL (2025-08-06 16:51:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
cted piece hash; context canceled; EOF
Aug 6 16:15:07.596899502 (2h6m15.535315351s) [noise=true , address=104.36.82.12:4151 , piece_id=B4D7VTLBQBRSPKA7N42KZTE7V5TSS754ZV7EF75JJEFDOW55O4CQ, seg_pos={0 0}, upload=2001]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Aug 6 16:15:07.596915132 (2h6m15.535330981s) [seg_pos={0 0}, upload=2001]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:15:07.596991619 (2h6m15.535407468s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:15:07.596995857 (2h6m15.535411706s) [upload=2001]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:15:07.887184563 (2h6m15.825600412s) [upload=2001]: Done uploading segments...
Aug 6 16:15:07.887203770 (2h6m15.825619609s) [upload=2001]: Upload finished. err: <nil>
Aug 6 16:16:00.579639900 (2h7m8.518055749s) [upload=2016]: Starting upload
Aug 6 16:16:00.579694645 (2h7m8.518110494s) [upload=2016]: Done starting upload
Aug 6 16:16:00.579789666 (2h7m8.518205515s) [upload=2016]: Uploading segments...
Aug 6 16:16:00.579861003 (2h7m8.518276852s) [upload=2016]: Got next segment. Inline: false
Aug 6 16:16:00.579885871 (2h7m8.51830172s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 6 16:16:00.579889858 (2h7m8.518305707s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 6 16:16:00.820626959 (2h7m8.759042808s) [seg_pos={0 0}, upload=2016]: Begin upload segment...
Aug 6 16:16:00.820694769 (2h7m8.759110629s) [seg_pos={0 0}, upload=2016]: Done begin upload segment.
Aug 6 16:16:00.820738192 (2h7m8.759154041s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.820754293 (2h7m8.759170142s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821224302 (2h7m8.759640151s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821272084 (2h7m8.759687983s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.821617053 (2h7m8.760032903s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822107232 (2h7m8.760523091s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822131258 (2h7m8.760547107s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822500845 (2h7m8.760916694s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822543386 (2h7m8.760959235s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.822973009 (2h7m8.761388858s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823204822 (2h7m8.761620661s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823426235 (2h7m8.761842084s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823693006 (2h7m8.762108855s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.823992970 (2h7m8.762408819s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824130774 (2h7m8.762546623s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:00.824422621 (2h7m8.76283848s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Uploading piece...
Aug 6 16:16:02.555639143 (2h7m10.494054992s) [upload=2016]: Next returned nil segment
Aug 6 16:16:02.555659893 (2h7m10.494075752s) [upload=2016]: Waiting for error group managing segments...
Aug 6 16:16:02.837308740 (2h7m10.775724589s) [noise=true , address=45.91.116.5:4298 , piece_id=WBH7PQI45FU2XVIE2BRQPLVCV32MME34DV24ZAQB625VGY3PDA3Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.882019636 (2h7m10.820435485s) [noise=true , address=77.90.31.19:4841 , piece_id=KQSHAVBEGRVWT7LHBNORUM2ZQBC6F5IQWQ3XCQ25UJUYNJW5QUEA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.917400741 (2h7m10.85581659s) [noise=true , address=45.91.116.8:4048 , piece_id=NAPMUXSXDIIWECP3QVA5CXSL7V4I2PRKEHD3422K6VCXI5YNIFMQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.947806114 (2h7m10.886221963s) [noise=true , address=45.91.116.12:4225 , piece_id=6ZDHNKA2TLCCVZ436KUG6MCWJN3BUWW34MR56AOWJI22XXVNBFQA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.964255246 (2h7m10.902671095s) [noise=true , address=104.36.82.4:4801 , piece_id=S5UFPPCIMSCXIAIF6PJ2P5L4KAA5IVO5RG4F33W3CAQA6KHVFR2Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972488429 (2h7m10.910904278s) [noise=true , address=104.36.82.9:4006 , piece_id=4BRGRC53TEW6FJRKN3U2ZE2JCNXJSP3EP7SPY22QNZUEX2WJQLPQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:02.972812389 (2h7m10.911228238s) [noise=true , address=45.91.116.14:4190 , piece_id=7OJP24AR7HKAPBY6ND33N4WZV2BWRVV4YIK6D6LCZ3LCR2QNZILQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.001575382 (2h7m10.939991231s) [noise=true , address=77.90.31.10:4520 , piece_id=XE4HQEP55KJKD7Z4AWBVG3ZY54ZL2SKUNASTUUZP6BAINDU6Z4PQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.028168973 (2h7m10.966584822s) [noise=true , address=104.36.82.4:4803 , piece_id=HDDNPPVS2TYDWNEKJO4UCI4KQB7VSBN55U4GRYDM6MLG2KC3QPCA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.036656182 (2h7m10.975072031s) [noise=true , address=45.91.116.7:4009 , piece_id=F3GL2GSZRKR3XXDK2PNCGYXIHRND6HAXGRMNGTKJVHUPO4JSIXBA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.044792840 (2h7m10.983208689s) [noise=true , address=104.36.82.12:4149 , piece_id=OKWNJAG5TT6LTJ7PQDMW26WBTBHKL6UVY467QMO5AFVL2TQPCFTQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.045187775 (2h7m10.983603634s) [noise=true , address=77.90.31.18:4805 , piece_id=75BZU4YOCX3VFJD4U3U3L2LKTRVULEQWNBI65OAO6PVLSGZXVIVA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118252640 (2h7m11.056668479s) [noise=true , address=104.36.82.5:4839 , piece_id=U3MUBVB72IHYEZ5LMTYKAQYEIUZSF7FO26SODUBKNU435AN3HKIQ, seg_pos={0 0}, upload=2016]: Done uploading piece. err: <nil>
Aug 6 16:16:03.118270995 (2h7m11.056686844s) [seg_pos={0 0}, upload=2016]: Segment reached optimal threshold of 13 pieces.
Aug 6 16:16:03.118456620 (2h7m11.056872469s) [noise=true , address=77.90.31.11:4553 , piece_id=BFHSMJA7CPQ23FR7G2ZFN3FQ44KOIIR6JCD23THQVKBRQLTHPGJA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119323248 (2h7m11.057739097s) [noise=true , address=104.36.82.9:4007 , piece_id=YNCBPP64M2CDK4R4K45T43CXVDAGU2AMVCDXTW4JGC4FEWF6PLHA, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 6 16:16:03.119734826 (2h7m11.058150675s) [noise=true , address=77.90.31.16:4733 , piece_id=6XP6XWV76DNPF5SWPPER7VFJMAAN5JT5NOVKBUKPNPJ5MXJSO74Q, seg_pos={0 0}, upload=2016]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33610->77.90.31.16:4733: use of closed network connection
Aug 6 16:16:03.119748902 (2h7m11.058164751s) [seg_pos={0 0}, upload=2016]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 6 16:16:03.119818285 (2h7m11.058234134s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 6 16:16:03.119823415 (2h7m11.058239264s) [upload=2016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 6 16:16:03.416354947 (2h7m11.354770796s) [upload=2016]: Done uploading segments...
Aug 6 16:16:03.416374936 (2h7m11.354790785s) [upload=2016]: Upload finished. err: <nil>
Duration: 5.864994053s
core
uplink
us-select-1
|