Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
15 out of 60 is failed (threshold=3) [.................................XXXXXX..X....XX.X..X.XX..XX] duration:
30.000524829s
FAIL (2025-09-06 12:45:29 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 30.000544526s
core
uplink
us1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
3 out of 60 is failed (threshold=3) [........................................X..X........X.......] duration:
1.328271642s
FAIL (2025-09-06 12:45:23 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 1.328295388s
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
14 out of 60 is failed (threshold=3) [.....................................X.XX.XXX.XXXX....XXX..X] duration:
30.000813071s
FAIL (2025-09-06 12:45:10 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 12:44:50.147066763 (190h36m46.827262322s) [upload=167655]: Starting upload Sep 6 12:44:50.147124814 (190h36m46.827320383s) [upload=167655]: Done starting upload Sep 6 12:44:50.147201060 (190h36m46.827396629s) [upload=167655]: Uploading segments... Sep 6 12:44:50.147233783 (190h36m46.827429352s) [upload=167655]: Got next segment. Inline: true Sep 6 12:44:50.147238792 (190h36m46.827434361s) [upload=167655]: Waiting for error group managing segments... Sep 6 12:44:50.147304157 (190h36m46.827499726s) [upload=167655]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 6 12:44:50.147308515 (190h36m46.827504084s) [upload=167655]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 6 12:44:50.147311211 (190h36m46.82750678s) [upload=167655]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 6 12:44:50.759036838 (190h36m47.439232417s) [upload=167655]: Done uploading segments... Sep 6 12:44:50.759058630 (190h36m47.439254199s) [upload=167655]: Upload finished. err: <nil>
Duration: 30.000824233s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
26 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.00168563s
FAIL (2025-09-06 12:45:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001710417s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
15 out of 60 is failed (threshold=3) [...........................................XXXXX.XX.XXXXXXXX] duration:
30.003079346s
FAIL (2025-09-06 12:45:06 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 30.003096118s
core
uplink
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
4 out of 60 is failed (threshold=3) [...................................X...X....X............X..] duration:
3.725881629s
FAIL (2025-09-06 12:45:05 UTC)
operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 44)
Duration: 3.725911166s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
9 out of 60 is failed (threshold=3) [......................................XX.X....XXX..XX.....X.] duration:
27.095411197s
FAIL (2025-09-06 12:45:04 UTC)
orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 6 12:43:41.208818309 (190h35m37.889013878s) [upload=167641]: Starting upload Sep 6 12:43:41.208890818 (190h35m37.889086387s) [upload=167641]: Done starting upload Sep 6 12:43:41.208969409 (190h35m37.889164978s) [upload=167641]: Uploading segments... Sep 6 12:43:41.209008523 (190h35m37.889204082s) [upload=167641]: Got next segment. Inline: true Sep 6 12:43:41.209012130 (190h35m37.889207689s) [upload=167641]: Waiting for error group managing segments... Sep 6 12:43:41.209044853 (190h35m37.889240422s) [upload=167641]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 6 12:43:41.209048781 (190h35m37.88924435s) [upload=167641]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 6 12:43:41.209051295 (190h35m37.889246864s) [upload=167641]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 6 12:44:19.603683397 (190h36m16.283878966s) [upload=167641]: Done uploading segments... Sep 6 12:44:19.603702664 (190h36m16.283898233s) [upload=167641]: Upload finished. err: <nil>
Duration: 27.095430273s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration:
1m1.46676597s
FAIL (2025-09-06 12:45:02 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
oise=true , address=145.79.227.44:4225 , piece_id=CENB2KFMH6MIETAI63JOGJHSUSARD6HVXRBPDP4OE2GFHXFGXD3Q, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:12HoBC3rCo4dTHYKsXZJNfSETHzkihdGfATJe46LdWM3DXVRdtR, address:145.79.227.44:4225): protocol: write tcp 188.166.225.170:45742->145.79.227.44:4225: use of closed network connection; write tcp 188.166.225.170:45742->145.79.227.44:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45742->145.79.227.44:4225: use of closed network connection
Sep 6 12:45:00.916180020 (190h36m57.596375589s) [noise=true , address=145.79.227.52:4464 , piece_id=I5EF5KWNIFGYFSCLNXQPCV7OARD7SCRHTEDPB3F2F6WBNM2JAV6Q, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk, address:145.79.227.52:4464): protocol: write tcp 188.166.225.170:59250->145.79.227.52:4464: use of closed network connection; write tcp 188.166.225.170:59250->145.79.227.52:4464: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59250->145.79.227.52:4464: use of closed network connection
Sep 6 12:45:00.916218853 (190h36m57.596414423s) [noise=true , address=145.79.227.3:4178 , piece_id=TG3CO4RFOUTKUBVN263FNOGDNTKWHBFJMFXNNEAQLWLEW4FDNWYA, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:12LjJ3SmaNTQi8U1rPSGmR461UYC2CfMtoZ22SSV97hDuhMwC4V, address:145.79.227.3:4178): protocol: read tcp 188.166.225.170:53652->145.79.227.3:4178: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:45:00.916265142 (190h36m57.596460711s) [noise=true , address=45.91.116.10:4093 , piece_id=X7HQ23R36V4NIWKQ76JCJUMAC6KB2DKAS2RPTESAE4R2LFKMSWWQ, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:12bHetxKYyr24K2FqogjTPcsq7soJUQK7Tq6tHFZtvrNAmQnAxZ, address:45.91.116.10:4093): protocol: read tcp 188.166.225.170:41802->45.91.116.10:4093: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:45:00.916317372 (190h36m57.596512941s) [noise=true , address=145.79.227.40:4078 , piece_id=XMNUXOMR6JKC7LGX6OE2FSAEI6MCUADTP6RPJOAKZQN3K4AL3MDA, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:1tzgtenV6us5ssD7xWGpZmVUoWEfbFhBNewbK9irHtZPkyDtRd, address:145.79.227.40:4078): protocol: write tcp 188.166.225.170:33638->145.79.227.40:4078: use of closed network connection; write tcp 188.166.225.170:33638->145.79.227.40:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33638->145.79.227.40:4078: use of closed network connection
Sep 6 12:45:00.916370224 (190h36m57.596565793s) [noise=true , address=145.79.227.14:4848 , piece_id=T5TVHIMZOGT4223RNZLHVX3E7Q5DIUDSMJ4MVSWYPWG7LAMLEW3A, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:1xvod7N8eEwunLDYWAvVUP1Cp1CdEr3quJUMG5kWyBrwzzr9vp, address:145.79.227.14:4848): protocol: write tcp 188.166.225.170:55140->145.79.227.14:4848: use of closed network connection; write tcp 188.166.225.170:55140->145.79.227.14:4848: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55140->145.79.227.14:4848: use of closed network connection
Sep 6 12:45:00.916411953 (190h36m57.596607522s) [noise=true , address=64.49.13.155:4920 , piece_id=ECVVVU7PNEHUPJZCYWARX44QC7MAKEBDZKURZYFQL7G3FREVL4UA, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:1frFBZNqMLDUjojkdhrBU4GFXckd3kB2Seok1yXngUT7UyciBz, address:64.49.13.155:4920): protocol: write tcp 188.166.225.170:37356->64.49.13.155:4920: use of closed network connection; write tcp 188.166.225.170:37356->64.49.13.155:4920: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37356->64.49.13.155:4920: use of closed network connection
Sep 6 12:45:00.916456168 (190h36m57.596651737s) [noise=true , address=145.79.227.7:4314 , piece_id=VPSDCXIZU3KQO74PLZXHYNN46KD5C2VGFQIZP276NR77LHFMVADA, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:12KjL9YES3swMrgFERTQczzsc62AcXVJXC3sFuHsSt5M4DHt5Xw, address:145.79.227.7:4314): protocol: write tcp 188.166.225.170:40444->145.79.227.7:4314: use of closed network connection; write tcp 188.166.225.170:40444->145.79.227.7:4314: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40444->145.79.227.7:4314: use of closed network connection
Sep 6 12:45:00.916489401 (190h36m57.59668497s) [noise=true , address=77.90.31.17:4791 , piece_id=UGR3ZCIJZORBPYJNKUQGOO3AQKJNOT62FD6EOK3MYQDPGFR54IYQ, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM, address:77.90.31.17:4791): protocol: write tcp 188.166.225.170:41280->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:41280->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41280->77.90.31.17:4791: use of closed network connection
Sep 6 12:45:00.916526211 (190h36m57.59672178s) [noise=true , address=145.79.227.12:4783 , piece_id=POLHEIBHQVPWSKDTCV7RO3FJNANA75AL3IBPTAZ7SIRFSY3634RA, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:12ito72D7vikRdjK7BPt4LnhAK84bzi8AbZZPNWrTqmbjwQT1N, address:145.79.227.12:4783): protocol: read tcp 188.166.225.170:48044->145.79.227.12:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:45:00.916532304 (190h36m57.596727873s) [noise=true , address=145.79.227.41:4115 , piece_id=XSFIJUAWZWJHXWIUJFYU6QX2ANVLHJQEEPISKWJJHJORWHBGH6WA, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:1ikQ2b343QztY3vBFQ3cmEwEEFszpB2TPMtstufHdFfMEkJkBV, address:145.79.227.41:4115): protocol: write tcp 188.166.225.170:36860->145.79.227.41:4115: use of closed network connection; write tcp 188.166.225.170:36860->145.79.227.41:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36860->145.79.227.41:4115: use of closed network connection
Sep 6 12:45:00.916559947 (190h36m57.596755516s) [noise=true , address=145.79.227.33:4894 , piece_id=X3ZTCS7IVJFJXRT7UGJYMYA7DSYSTPVFEBIXDH6GG6UYZUKNOFJQ, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44, address:145.79.227.33:4894): protocol: write tcp 188.166.225.170:46842->145.79.227.33:4894: use of closed network connection; write tcp 188.166.225.170:46842->145.79.227.33:4894: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46842->145.79.227.33:4894: use of closed network connection
Sep 6 12:45:00.916597628 (190h36m57.596793197s) [noise=true , address=145.79.227.2:4141 , piece_id=DMFXUMHATHAG3LYLY6SXQSEJNCIBYSNVLVUNH3C65YQFYWVKTS4Q, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:129zfpWSguxcUH9723ivdoL39WgpjiTxA189mYbzsGAGVjFcXLL, address:145.79.227.2:4141): protocol: write tcp 188.166.225.170:54318->145.79.227.2:4141: use of closed network connection; write tcp 188.166.225.170:54318->145.79.227.2:4141: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54318->145.79.227.2:4141: use of closed network connection
Sep 6 12:45:00.916604361 (190h36m57.59679993s) [noise=true , address=45.91.116.5:4305 , piece_id=EF2HEIGYINOHDXBDWERMHSF5RKEBEAJUAI2MI35PPKZEEDNLWJGA, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:121EfsJiXbm2HktqinYqjYaHWRyGiu2wA9fq4PQXpyizYttDCi1, address:45.91.116.5:4305): protocol: read tcp 188.166.225.170:34222->45.91.116.5:4305: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:45:00.916617847 (190h36m57.596813416s) [noise=true , address=145.79.227.37:4571 , piece_id=4V5GDKW4CESRWBP33OZYOLT37D6CMOPMOAVQYEU4I7GDL5UE3CZA, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:145.79.227.37:4571): protocol: write tcp 188.166.225.170:35714->145.79.227.37:4571: use of closed network connection; write tcp 188.166.225.170:35714->145.79.227.37:4571: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35714->145.79.227.37:4571: use of closed network connection
Sep 6 12:45:00.916660078 (190h36m57.596855647s) [noise=true , address=45.91.116.9:4278 , piece_id=I4USVTGJF5GDSWPU5I6WEWEBJFAOXLYD7ABLCBO7YW6KAINKDLAA, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:12htsH8bL33XpUeEstKHb2yeL6TNLusbdRmms9bwzmVCHkej2Xo, address:45.91.116.9:4278): protocol: write tcp 188.166.225.170:59172->45.91.116.9:4278: use of closed network connection; write tcp 188.166.225.170:59172->45.91.116.9:4278: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59172->45.91.116.9:4278: use of closed network connection
Sep 6 12:45:00.916674295 (190h36m57.596869864s) [noise=true , address=145.79.227.51:4431 , piece_id=TML3VPN426RQ2ME74KFH3JZM25O6UMFKRQ2XA56H2FNK3LWCURAA, seg_pos={0 0}, upload=167645]: Done uploading piece. err: ecclient: upload failed (node:1KBe2tkkC8f4gXvHFRiZNNaUYoVVd6scwbxWDDBCbcZJPd5kbj, address:145.79.227.51:4431): protocol: read tcp 188.166.225.170:43510->145.79.227.51:4431: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:45:00.916716876 (190h36m57.596912445s) [seg_pos={0 0}, upload=167645]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 1m1.466781921s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
7 out of 60 is failed (threshold=3) [.....................................X......X..XXXX.....X...] duration:
5.348076241s
FAIL (2025-09-06 12:44:58 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 48)
Duration: 5.348121708s
core
uplink
eu1
|
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)antiblip:
4 out of 60 is failed (threshold=3) [..................................XX.......X......X.........] duration:
860.38228ms target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-09-06 12:44:57 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout
Duration: 860.414432ms
edge
linksharing
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [..................................................X.X.......] duration:
22.364166963s remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-09-06 12:44:56 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 22.364193485s
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
14 out of 60 is failed (threshold=3) [..................................XXXXXX..X....XX.X..X.XX..X] duration:
30.001054151s
FAIL (2025-09-06 12:44:48 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.001072976s
core
uplink
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
5 out of 60 is failed (threshold=3) [...................................XX.X.X.X.................] duration:
8.712460812s target:
https://us1.storj.io/login
FAIL (2025-09-06 12:44:48 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout
Duration: 8.712472564s
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
22 out of 60 is failed (threshold=3) [..................................XXXXXXX.X.XXXXXXXXXXXXXX..] duration:
15.618239609s
FAIL (2025-09-06 12:44:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 15.618260098s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
13 out of 60 is failed (threshold=3) [......................................X.XX.XXX.XXXX....XXX..] duration:
28.789342841s
FAIL (2025-09-06 12:44:39 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 28.789378068s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration:
1m0.178011439s
FAIL (2025-09-06 12:44:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 6)
======LOGS======
ion; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:44:38.926231071 (190h36m35.60642669s) [noise=true , address=185.167.204.206:4062 , piece_id=MTJLXNYTG3ZJRMDNWDURTZH7WQA56V4N33M4S67XCTEULZFBE3IA, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 188.166.225.170:55340->185.167.204.206:4062: use of closed network connection; write tcp 188.166.225.170:55340->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55340->185.167.204.206:4062: use of closed network connection
Sep 6 12:44:38.926285315 (190h36m35.606480884s) [noise=true , address=185.167.204.206:4047 , piece_id=X7MXXLMMQ5NQLSBYURD52HBVEO3PSTG7R3LKCKATJPFZHRWMFJ2A, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: read tcp 188.166.225.170:46966->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:44:38.926292148 (190h36m35.606487717s) [noise=true , address=185.167.204.206:4048 , piece_id=Y5TZAODLZGLWLBFWRAMYRMFOJEOELH7PSYLTSRAQC4M4WP5LSIKA, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 188.166.225.170:38580->185.167.204.206:4048: use of closed network connection; write tcp 188.166.225.170:38580->185.167.204.206:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38580->185.167.204.206:4048: use of closed network connection
Sep 6 12:44:38.926302378 (190h36m35.606497947s) [noise=true , address=185.167.204.206:4063 , piece_id=NPNBDS2SN767OPQHC7GUC43P2V3TWZ3Q2BVNCEMLH5PKBVZB5JHA, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: read tcp 188.166.225.170:43212->185.167.204.206:4063: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:44:38.926330823 (190h36m35.606526392s) [noise=true , address=185.167.204.205:4005 , piece_id=N4356TMYCMDS4TUD5CXQK7OUTTNDC2EXEGKZZGTFBYOKOS7TRIDQ, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, address:185.167.204.205:4005): protocol: read tcp 188.166.225.170:56964->185.167.204.205:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:44:38.926468576 (190h36m35.606664145s) [noise=true , address=81.171.14.70:4042 , piece_id=AGWD2GKCEJVC3CT7VMVDVKNHYQWKJLVPW5MDP6ZVSFROYVBWA23Q, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: expected piece hash; write tcp 188.166.225.170:57226->81.171.14.70:4042: use of closed network connection; write tcp 188.166.225.170:57226->81.171.14.70:4042: use of closed network connection; EOF
Sep 6 12:44:38.926489326 (190h36m35.606684895s) [noise=true , address=81.171.14.70:4036 , piece_id=DKSJ4AS5QA4NFMTUQGSS3KIWCRNO5QPDF2X4IU4XWNPTJNGW63RA, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: expected piece hash; context deadline exceeded; EOF
Sep 6 12:44:38.926564199 (190h36m35.606759768s) [noise=true , address=185.167.204.205:4036 , piece_id=EP26RJGWSTT7QXJRJEUD35WATBC542BXAE6HOKLIY6ZDMOERIG4Q, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:12Mngc2aRWoo1ZQGktFfA4DauhKyEFcbRk6dg7nYa1jZUg2srMV, address:185.167.204.205:4036): protocol: write tcp 188.166.225.170:47764->185.167.204.205:4036: use of closed network connection; write tcp 188.166.225.170:47764->185.167.204.205:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->185.167.204.205:4036: use of closed network connection
Sep 6 12:44:38.926600469 (190h36m35.606796038s) [noise=true , address=81.171.14.70:4032 , piece_id=XT4XMCG7K5HOCLU4M3KTXEVVP4GS6YXC4I54WTWCPPL6EAIE3MYQ, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:52914->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:52914->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52914->81.171.14.70:4032: use of closed network connection
Sep 6 12:44:38.926632179 (190h36m35.606827738s) [noise=true , address=81.171.14.70:4041 , piece_id=K5ZROFLPISKMIPBDF3SYBAYBY22R3ZA2G2GON57C3JFKFFUSPKRQ, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:37852->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:37852->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->81.171.14.70:4041: use of closed network connection
Sep 6 12:44:38.926712032 (190h36m35.606907601s) [noise=true , address=185.167.204.205:4031 , piece_id=QWB5QKQ55KQY5AM2O225V2FTKGIGIZJSN6EE5X56YJU2ER43NWHQ, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR, address:185.167.204.205:4031): protocol: write tcp 188.166.225.170:55686->185.167.204.205:4031: use of closed network connection; write tcp 188.166.225.170:55686->185.167.204.205:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55686->185.167.204.205:4031: use of closed network connection
Sep 6 12:44:38.926729645 (190h36m35.606925214s) [noise=true , address=185.167.204.205:4022 , piece_id=ULGLGBDGMPZQCKU5OMCOA3PERYS43P4WMSOWYMLC4MWDRMNEBMUQ, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: write tcp 188.166.225.170:59286->185.167.204.205:4022: use of closed network connection; write tcp 188.166.225.170:59286->185.167.204.205:4022: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->185.167.204.205:4022: use of closed network connection
Sep 6 12:44:38.926774782 (190h36m35.606970351s) [noise=true , address=185.167.204.206:4057 , piece_id=S22E4UYGL4LNX2KKQ2DAML47J2JF6ISHJP445L7IZCYYHYIXNYIA, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: read tcp 188.166.225.170:51770->185.167.204.206:4057: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:44:38.926782807 (190h36m35.606978376s) [noise=true , address=185.167.204.206:4056 , piece_id=MWLMKX2LXNEPCTEI2JDC36NMEXM5HNBBJKJONKKD7OICCX5ACG5A, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: write tcp 188.166.225.170:52318->185.167.204.206:4056: use of closed network connection; write tcp 188.166.225.170:52318->185.167.204.206:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52318->185.167.204.206:4056: use of closed network connection
Sep 6 12:44:38.926808957 (190h36m35.607004526s) [noise=true , address=185.167.204.205:4008 , piece_id=QN3S6ITGRK6RPS3VE3WODXDQFU3XUAL46JP5UF3EVCRTF3LEK3HQ, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 188.166.225.170:59458->185.167.204.205:4008: use of closed network connection; write tcp 188.166.225.170:59458->185.167.204.205:4008: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59458->185.167.204.205:4008: use of closed network connection
Sep 6 12:44:38.926856398 (190h36m35.607051967s) [noise=true , address=185.167.204.205:4002 , piece_id=IAHFVFS6PJYHIWOHZFN5EQOMYV4LUTWBJRR2BAANPYDR5MXW7BKQ, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:122KLY1enErmKthxbRteifu9eqBKwRCwyvmFdM1yErYQnXQpxQ3, address:185.167.204.205:4002): protocol: read tcp 188.166.225.170:46632->185.167.204.205:4002: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:44:38.926885073 (190h36m35.607080642s) [noise=true , address=185.167.204.206:4052 , piece_id=WPE22AH3LB7M3GG6L2HRUCGC3MZBA2JB7GLSQVRWNGSTPO25GO6Q, seg_pos={0 0}, upload=167639]: Done uploading piece. err: ecclient: upload failed (node:12Xr3UmaAcshmxZMecu5i7VHA6LJxiZm9u6wngapEma3hz3tNmn, address:185.167.204.206:4052): protocol: read tcp 188.166.225.170:41680->185.167.204.206:4052: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:44:38.926907606 (190h36m35.607103175s) [seg_pos={0 0}, upload=167639]: Done waiting for segment. successful: 6 optimal: 26 errs: <nil>
Sep 6 12:44:39.102218459 (190h36m35.782414029s) [upload=167639]: Done uploading segments...
Sep 6 12:44:39.102242736 (190h36m35.782438305s) [upload=167639]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 6)
Duration: 1m0.178024343s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
5 out of 60 is failed (threshold=3) [....................................XX.X.X.X................] duration:
23.463655652s target:
https://us1.storj.io/login
FAIL (2025-09-06 12:44:39 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout
Duration: 23.463680851s
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration:
1m0.000965647s
FAIL (2025-09-06 12:44:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000990434s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
3 out of 60 is failed (threshold=3) [.........................................X..X........X......] duration:
20.414971954s
FAIL (2025-09-06 12:44:38 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 20.415080893s
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
9 out of 60 is failed (threshold=3) [.......................................XX.X....XXX..XX.....X] duration:
1m0.001532199s
FAIL (2025-09-06 12:44:37 UTC)
orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 6 12:43:41.208818309 (190h35m37.889013878s) [upload=167641]: Starting upload Sep 6 12:43:41.208890818 (190h35m37.889086387s) [upload=167641]: Done starting upload Sep 6 12:43:41.208969409 (190h35m37.889164978s) [upload=167641]: Uploading segments... Sep 6 12:43:41.209008523 (190h35m37.889204082s) [upload=167641]: Got next segment. Inline: true Sep 6 12:43:41.209012130 (190h35m37.889207689s) [upload=167641]: Waiting for error group managing segments... Sep 6 12:43:41.209044853 (190h35m37.889240422s) [upload=167641]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 6 12:43:41.209048781 (190h35m37.88924435s) [upload=167641]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 6 12:43:41.209051295 (190h35m37.889246864s) [upload=167641]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 6 12:44:19.603683397 (190h36m16.283878966s) [upload=167641]: Done uploading segments... Sep 6 12:44:19.603702664 (190h36m16.283898233s) [upload=167641]: Upload finished. err: <nil>
Duration: 1m0.001549764s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
14 out of 60 is failed (threshold=3) [............................................XXXXX.XX.XXXXXXX] duration:
33.788116488s
FAIL (2025-09-06 12:44:36 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 33.788137167s
core
uplink
us1
|
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)antiblip:
4 out of 60 is failed (threshold=3) [...................................XX.......X......X........] duration:
844.814861ms target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-09-06 12:44:21 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout
Duration: 844.836393ms
edge
linksharing
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
7 out of 60 is failed (threshold=3) [......................................X......X..XXXX.....X..] duration:
5.372708696s
FAIL (2025-09-06 12:44:18 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 48)
Duration: 5.372740055s
core
uplink
eu1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
4 out of 60 is failed (threshold=3) [....................................X...X....X............X.] duration:
4.192776585s
FAIL (2025-09-06 12:44:10 UTC)
operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 44)
Duration: 4.192803466s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
4 out of 60 is failed (threshold=3) [.....................................X...X....X............X] duration:
30.020448654s
FAIL (2025-09-06 12:44:05 UTC)
operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 44)
Duration: 30.020473974s
core
uplink
ap1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
5 out of 60 is failed (threshold=3) [.....................................XX.X.X.X...............] duration:
6.773337027s target:
https://us1.storj.io/login
FAIL (2025-09-06 12:44:01 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout
Duration: 6.773363337s
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration:
3m4.961586113s
FAIL (2025-09-06 12:44:00 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
upload=167608]: Done uploading piece. err: ecclient: upload failed (node:16EACrmREmTxBUPQKwsABtESUoZKWzcxKCknE81xK1nrDYpZSg, address:77.90.31.18:4822): protocol: write tcp 188.166.225.170:38434->77.90.31.18:4822: use of closed network connection; write tcp 188.166.225.170:38434->77.90.31.18:4822: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38434->77.90.31.18:4822: use of closed network connection
Sep 6 12:41:55.704824229 (190h33m52.385019798s) [noise=true , address=77.90.31.17:4785 , piece_id=FOJJM5ORNSOLNE7RI7UUC6H2LODYNM4YCHINKJKASXHH54CY2OGA, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:1P9AKP1rxpVvkxDs6XH2S6hR8DRHBZkWMZiYPTcZZjMZwLAaBN, address:77.90.31.17:4785): protocol: write tcp 188.166.225.170:35572->77.90.31.17:4785: use of closed network connection; write tcp 188.166.225.170:35572->77.90.31.17:4785: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35572->77.90.31.17:4785: use of closed network connection
Sep 6 12:41:55.704867493 (190h33m52.385063062s) [noise=true , address=145.79.227.51:4431 , piece_id=PVKLSJAQAUKLYGMU5PF6HPKENTCDMMCA5EHSUTHBSFDZTKCEPIIQ, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:1KBe2tkkC8f4gXvHFRiZNNaUYoVVd6scwbxWDDBCbcZJPd5kbj, address:145.79.227.51:4431): protocol: write tcp 188.166.225.170:47032->145.79.227.51:4431: use of closed network connection; write tcp 188.166.225.170:47032->145.79.227.51:4431: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47032->145.79.227.51:4431: use of closed network connection
Sep 6 12:41:55.704899934 (190h33m52.385095503s) [noise=true , address=145.79.227.48:4306 , piece_id=VMQCEWZ3ABFE46YILSU6UGRQKNMIRQLKOQYT7IDFP3O3FOMGVJ7Q, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw, address:145.79.227.48:4306): protocol: write tcp 188.166.225.170:53468->145.79.227.48:4306: use of closed network connection; write tcp 188.166.225.170:53468->145.79.227.48:4306: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53468->145.79.227.48:4306: use of closed network connection
Sep 6 12:41:55.704933819 (190h33m52.385129378s) [noise=true , address=45.91.116.6:4990 , piece_id=D5BSOBZJVW55SJIJGFUITWDD3PNOSCKFHZHEMAH3O4AXMNPXISEQ, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:126JXfJvuad1JedXjgEJuA12awNtwHBYNZhHSJoei2ZYTrccDmA, address:45.91.116.6:4990): protocol: write tcp 188.166.225.170:54750->45.91.116.6:4990: use of closed network connection; write tcp 188.166.225.170:54750->45.91.116.6:4990: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54750->45.91.116.6:4990: use of closed network connection
Sep 6 12:41:55.704967444 (190h33m52.385163013s) [noise=true , address=145.79.227.31:4813 , piece_id=I5OZBEW67FWWL5M42Z3UBFYGS6FUSP2FAX4QIKYFCCO5OA6P4HUQ, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:1Egk1BCBmtjUZcwJKpoQjfrubA2czJKMEai5bzDqkPmXTRhaB6, address:145.79.227.31:4813): protocol: write tcp 188.166.225.170:51480->145.79.227.31:4813: use of closed network connection; write tcp 188.166.225.170:51480->145.79.227.31:4813: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51480->145.79.227.31:4813: use of closed network connection
Sep 6 12:41:55.705009394 (190h33m52.385204963s) [noise=true , address=145.79.227.40:4051 , piece_id=2ZASNO3ZQSVPJPKH7X4OCTFPTIZTSBIFUDYWITEEYRITW5UHIZ6A, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: write tcp 188.166.225.170:47374->145.79.227.40:4051: use of closed network connection; write tcp 188.166.225.170:47374->145.79.227.40:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47374->145.79.227.40:4051: use of closed network connection
Sep 6 12:41:55.705090720 (190h33m52.385286289s) [noise=true , address=145.79.227.11:4742 , piece_id=5OFTJ4OC2TSVAZHOXVZIL4ZX4AV3RS4LJGZFLQRE3VJBMGTQ6PQA, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:1WsGo4te3PENFo9YRioYFrhkeRQvztnLVZJV4i38TV3TjKqBFT, address:145.79.227.11:4742): protocol: write tcp 188.166.225.170:56974->145.79.227.11:4742: use of closed network connection; write tcp 188.166.225.170:56974->145.79.227.11:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56974->145.79.227.11:4742: use of closed network connection
Sep 6 12:41:55.705101480 (190h33m52.385297049s) [upload=167608]: Next segment err: context deadline exceeded
Sep 6 12:41:55.705111069 (190h33m52.385306638s) [noise=true , address=145.79.227.42:4126 , piece_id=UEWVJJHXTTAO5S2JN5HGP3DYRZZT6RS7WT6PYTCDQZWGKXLABQYQ, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:145.79.227.42:4126): protocol: read tcp 188.166.225.170:42480->145.79.227.42:4126: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:41:55.705152497 (190h33m52.385348067s) [noise=true , address=145.79.227.32:4838 , piece_id=M6SFH3HXJ3OSBDMQM2EHBQY6ANUVYIDPSX77X3U7VIHIEGOJOFTA, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:1eorQLNmgEaXv8myr6t3Q8rysGwsvtdZzu5b7sndhoANb6Waj6, address:145.79.227.32:4838): protocol: read tcp 188.166.225.170:35156->145.79.227.32:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:41:55.705188036 (190h33m52.385383615s) [noise=true , address=145.79.227.17:4001 , piece_id=ZDLO5TLQ7YIW2DGOD4NJYTBD6F5ZVO2TBUOVITWKH7X6S3L6QDMQ, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:12Eaef5RrMHiZS6dE7wTXyq239F1RXovU7UCTQ7uC6Qusj6bJDw, address:145.79.227.17:4001): protocol: write tcp 188.166.225.170:38056->145.79.227.17:4001: use of closed network connection; write tcp 188.166.225.170:38056->145.79.227.17:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38056->145.79.227.17:4001: use of closed network connection
Sep 6 12:41:55.705246027 (190h33m52.385441596s) [noise=true , address=145.79.227.3:4178 , piece_id=PHFMCGI2K5VTEB3GPGXFKELFYO2JE63LD644NLDYI4ODAF5AO5MA, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:12LjJ3SmaNTQi8U1rPSGmR461UYC2CfMtoZ22SSV97hDuhMwC4V, address:145.79.227.3:4178): protocol: write tcp 188.166.225.170:39886->145.79.227.3:4178: use of closed network connection; write tcp 188.166.225.170:39886->145.79.227.3:4178: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39886->145.79.227.3:4178: use of closed network connection
Sep 6 12:41:55.705280483 (190h33m52.385476062s) [noise=true , address=145.79.227.29:4974 , piece_id=6AHIGJUGFP2ELYMMWZU72GT5P2HRQEQM4NYUAAPEJ4NNTU52SL3A, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:127knbpTBAt5v44Kbz5tYDYfZhmXPa8tBaV43emRXkmKh2E5XMp, address:145.79.227.29:4974): protocol: write tcp 188.166.225.170:43568->145.79.227.29:4974: use of closed network connection; write tcp 188.166.225.170:43568->145.79.227.29:4974: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43568->145.79.227.29:4974: use of closed network connection
Sep 6 12:41:55.705318395 (190h33m52.385513964s) [noise=true , address=145.79.227.14:4863 , piece_id=2BPSDK65ZZUKKF7J4X2KFYQGDBAI2RWNA4LLIHZ7JGCHIYNES3UA, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn, address:145.79.227.14:4863): protocol: read tcp 188.166.225.170:39714->145.79.227.14:4863: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:41:55.705365716 (190h33m52.385561295s) [noise=true , address=145.79.227.36:4536 , piece_id=7A7DT4O65XHUG66RQ3FABWXBMN2V2FXNILMLMJOHNL4QIUHUBMPQ, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:1WoVNAp5RfVviaxN4RWCUDEhxLKsdGB8bgE6gcUcgZKcM4Cz1s, address:145.79.227.36:4536): protocol: write tcp 188.166.225.170:50390->145.79.227.36:4536: use of closed network connection; write tcp 188.166.225.170:50390->145.79.227.36:4536: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50390->145.79.227.36:4536: use of closed network connection
Sep 6 12:41:55.705377439 (190h33m52.385573008s) [noise=true , address=145.79.227.37:4586 , piece_id=WCQL4N5MTPKLI7O3YZP5XWDRZZZE2X22NXLOKJ7JBOJF46P2EY5A, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:12YdSTc2X61baRotZGev4q25BFkEspbienJtMcpbQvGgbyCRYfx, address:145.79.227.37:4586): protocol: read tcp 188.166.225.170:47122->145.79.227.37:4586: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:41:55.705439086 (190h33m52.385634655s) [noise=true , address=145.79.227.35:4494 , piece_id=WTITHZZCFMZFBBO2FTAUGIN2DN7XV2FO6DIAH2YFN2LKW26NS5YA, seg_pos={0 0}, upload=167608]: Done uploading piece. err: ecclient: upload failed (node:17b8LdpavJCCFXEBj7ndUKR6sge6Ntv96cGK5TmahF3MtwTV9z, address:145.79.227.35:4494): protocol: read tcp 188.166.225.170:42878->145.79.227.35:4494: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:41:55.705462161 (190h33m52.38565774s) [seg_pos={0 0}, upload=167608]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 3m4.961608154s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
25 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.000376074s
FAIL (2025-09-06 12:43:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000395904s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
13 out of 60 is failed (threshold=3) [...................................XXXXXX..X....XX.X..X.XX..] duration:
23.8348722s
FAIL (2025-09-06 12:43:55 UTC)
orbiter warning: operation: DownloadObject
uplink: object not found ("uplink-tests/from_do_prod_ops_orbiter_ap_0/1K")
Duration: 23.834899671s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
22 out of 60 is failed (threshold=3) [...................................XXXXXXX.X.XXXXXXXXXXXXXX.] duration:
24.409284022s
FAIL (2025-09-06 12:43:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 24.409308729s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
13 out of 60 is failed (threshold=3) [.............................................XXXXX.XX.XXXXXX] duration:
30.000662068s
FAIL (2025-09-06 12:43:49 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 30.000676266s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [...................................................X.X......] duration:
14.157610136s remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-09-06 12:43:45 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 14.157633619s
edge
gateway
vhost
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
3 out of 60 is failed (threshold=3) [..........................................X..X........X.....] duration:
6.494404057s
FAIL (2025-09-06 12:43:42 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 6.494423803s
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
7 out of 60 is failed (threshold=3) [.......................................X......X..XXXX.....X.] duration:
5.664664743s
FAIL (2025-09-06 12:43:40 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 48)
Duration: 5.664695261s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
5 out of 60 is failed (threshold=3) [......................................XX.X.X.X..............] duration:
4.04704761s target:
https://us1.storj.io/login
FAIL (2025-09-06 12:43:39 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout
Duration: 4.047111222s
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration:
1m0.001149439s
FAIL (2025-09-06 12:43:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001174477s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration:
1m0.173834979s
FAIL (2025-09-06 12:43:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 6)
======LOGS======
.166.225.170:59386->81.171.14.70:4031: use of closed network connection
Sep 6 12:43:38.501326034 (190h35m35.181521603s) [noise=true , address=185.167.204.205:4002 , piece_id=TEZNOIDRD6KFI6XKD4YRCYVREQSOAEDSI6MSGG6ARTCJH3S7JP5Q, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:122KLY1enErmKthxbRteifu9eqBKwRCwyvmFdM1yErYQnXQpxQ3, address:185.167.204.205:4002): protocol: write tcp 188.166.225.170:55538->185.167.204.205:4002: use of closed network connection; write tcp 188.166.225.170:55538->185.167.204.205:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55538->185.167.204.205:4002: use of closed network connection
Sep 6 12:43:38.501510107 (190h35m35.181705676s) [noise=true , address=185.167.204.206:4064 , piece_id=NRUOCJA5BQP3KX77YTPFIFCDOHGGRWMJBQ3YMTG3AZV6NW26EVBQ, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:1ZB3emcQntFxpr7zBiE98eesnjoTnNKinw3FPtGU2TgznGakeb, address:185.167.204.206:4064): protocol: write tcp 188.166.225.170:46466->185.167.204.206:4064: use of closed network connection; write tcp 188.166.225.170:46466->185.167.204.206:4064: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->185.167.204.206:4064: use of closed network connection
Sep 6 12:43:38.501542017 (190h35m35.181737587s) [noise=true , address=185.167.204.205:4025 , piece_id=E7X6RFYMIWOZB44P6NDFTW5RYHIMKQG5YZMUZN3LE4PXGRMMLKIA, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: write tcp 188.166.225.170:53776->185.167.204.205:4025: use of closed network connection; write tcp 188.166.225.170:53776->185.167.204.205:4025: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53776->185.167.204.205:4025: use of closed network connection
Sep 6 12:43:38.501547898 (190h35m35.181743468s) [noise=true , address=185.167.204.206:4049 , piece_id=DO4W2GBTM7XGI66ESH4OWKM53F62PRPHQSH5UT5G5IUH7XR2YXVA, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:38566->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:38566->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38566->185.167.204.206:4049: use of closed network connection
Sep 6 12:43:38.501584920 (190h35m35.181780489s) [noise=true , address=185.167.204.206:4057 , piece_id=MPBMWBAGG7GQ3MHB2A6FZFBJVCTIZ4GG3QMVVQBUHRSJ5RXSZLVA, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: write tcp 188.166.225.170:57782->185.167.204.206:4057: use of closed network connection; write tcp 188.166.225.170:57782->185.167.204.206:4057: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57782->185.167.204.206:4057: use of closed network connection
Sep 6 12:43:38.501598425 (190h35m35.181793994s) [noise=true , address=185.167.204.206:4053 , piece_id=J27MQY5GIGH5BGBWBY4USVR3QN6IAGNPC5OZU5H7VIRIP7UCBGBQ, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: write tcp 188.166.225.170:53880->185.167.204.206:4053: use of closed network connection; write tcp 188.166.225.170:53880->185.167.204.206:4053: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53880->185.167.204.206:4053: use of closed network connection
Sep 6 12:43:38.501622792 (190h35m35.181818361s) [noise=true , address=185.167.204.205:4024 , piece_id=MJGPLRHD7CVZ7K6IBREX5OQWQJVZWTNFH7J7QTDKGSJZAQ5QZNVQ, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 188.166.225.170:56508->185.167.204.205:4024: use of closed network connection; write tcp 188.166.225.170:56508->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56508->185.167.204.205:4024: use of closed network connection
Sep 6 12:43:38.501655184 (190h35m35.181850753s) [noise=true , address=81.171.14.70:4032 , piece_id=VDPG4KMK3OYJM3CTRWMGS6N67YIHYH2S4F3MW2FBFS4AC26AQD2Q, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:60298->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:60298->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60298->81.171.14.70:4032: use of closed network connection
Sep 6 12:43:38.501685442 (190h35m35.181881011s) [noise=true , address=185.167.204.205:4036 , piece_id=ZT6OLH2BPXFCPOURAP4DIENWHLUWGO7NJ5XUTNA6M637QU3OBMNQ, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:12Mngc2aRWoo1ZQGktFfA4DauhKyEFcbRk6dg7nYa1jZUg2srMV, address:185.167.204.205:4036): protocol: read tcp 188.166.225.170:54052->185.167.204.205:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:43:38.501706011 (190h35m35.18190158s) [noise=true , address=185.167.204.205:4004 , piece_id=SOXIBQZC3OPJLQL4LJ5AJ5CBSKSYHMWL7EWLUGV6WPCQJ3LHDVYA, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 188.166.225.170:33356->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:33356->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33356->185.167.204.205:4004: use of closed network connection
Sep 6 12:43:38.501716872 (190h35m35.181912441s) [noise=true , address=185.167.204.206:4070 , piece_id=XPHE53GAOL3XWPIZOAVBPU5P7TTKXAEQ33DSTBX42KCKIYUGBS6A, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:1HjpnutE3cSKeeVw3nEvoPPAbJKjP6VZYMDGLeT9zSskkwzBUz, address:185.167.204.206:4070): protocol: read tcp 188.166.225.170:53132->185.167.204.206:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:43:38.501742200 (190h35m35.181937769s) [noise=true , address=81.171.14.70:4037 , piece_id=IT6XHIMETJ5I43NDPUUNDLQCYGOZGVTHWSV5ZZPMJUQBVHQQVJ6A, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:51814->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:51814->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51814->81.171.14.70:4037: use of closed network connection
Sep 6 12:43:38.501789741 (190h35m35.18198531s) [noise=true , address=185.167.204.205:4008 , piece_id=YNTXB4334IE3P5WXSYBUJEUKP3C2LFO2KZ5P6FCHZF2PQMRADMAA, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 188.166.225.170:60664->185.167.204.205:4008: use of closed network connection; write tcp 188.166.225.170:60664->185.167.204.205:4008: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60664->185.167.204.205:4008: use of closed network connection
Sep 6 12:43:38.501829889 (190h35m35.182025458s) [noise=true , address=81.171.14.70:4033 , piece_id=RPL2YSPKKW7WUC27ZHKNAHZL4M7XXXM23VCXK5UTH65C3JZPF7VQ, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:46708->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:43:38.501839036 (190h35m35.182034605s) [noise=true , address=185.167.204.206:4063 , piece_id=4B3USL2XQKDRWS62IMTT72LBNL6Y4YMHMDVKNYMKULFUDHQ7CZUQ, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: write tcp 188.166.225.170:46102->185.167.204.206:4063: use of closed network connection; write tcp 188.166.225.170:46102->185.167.204.206:4063: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46102->185.167.204.206:4063: use of closed network connection
Sep 6 12:43:38.501880896 (190h35m35.182076465s) [noise=true , address=185.167.204.206:4058 , piece_id=3W3FXFWS2ORB3FS4TRCJYRDDCWCNX22ZAACQAOWMR7WPNLD2W6MQ, seg_pos={0 0}, upload=167628]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: read tcp 188.166.225.170:58116->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 6 12:43:38.501900093 (190h35m35.182095652s) [seg_pos={0 0}, upload=167628]: Done waiting for segment. successful: 6 optimal: 26 errs: <nil>
Sep 6 12:43:38.673234627 (190h35m35.353430196s) [upload=167628]: Done uploading segments...
Sep 6 12:43:38.673266999 (190h35m35.353462568s) [upload=167628]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 6)
Duration: 1m0.173847203s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)antiblip:
4 out of 60 is failed (threshold=3) [....................................XX.......X......X.......] duration:
17.211651806s target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-09-06 12:43:36 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout
Duration: 17.211715358s
edge
linksharing
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
3 out of 60 is failed (threshold=3) [...........................................X..X........X....] duration:
1m25.145078799s
FAIL (2025-09-06 12:43:36 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 1m25.145102644s
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
3 out of 60 is failed (threshold=3) [......................................X...X....X............] duration:
3.413190475s
FAIL (2025-09-06 12:43:22 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 47)
Duration: 3.413221133s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
12 out of 60 is failed (threshold=3) [..............................................XXXXX.XX.XXXXX] duration:
1m14.115578434s
FAIL (2025-09-06 12:43:18 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 1m14.115596008s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
13 out of 60 is failed (threshold=3) [.......................................X.XX.XXX.XXXX....XXX.] duration:
16.236088661s
FAIL (2025-09-06 12:43:17 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 16.236123669s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [....................................................X.X.....] duration:
15.303658967s remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-09-06 12:43:12 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 15.30368139s
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
8 out of 60 is failed (threshold=3) [........................................XX.X....XXX..XX.....] duration:
14.457658179s
FAIL (2025-09-06 12:43:09 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 14.457676224s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
22 out of 60 is failed (threshold=3) [....................................XXXXXXX.X.XXXXXXXXXXXXXX] duration:
30.000538044s
FAIL (2025-09-06 12:43:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000562582s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
7 out of 60 is failed (threshold=3) [........................................X......X..XXXX.....X] duration:
30.181899647s
FAIL (2025-09-06 12:43:03 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 48)
Duration: 30.181927218s
core
uplink
eu1
|
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)antiblip:
4 out of 60 is failed (threshold=3) [.....................................XX.......X......X......] duration:
842.202813ms target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-09-06 12:43:03 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout
Duration: 842.226789ms
edge
linksharing
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
8 out of 60 is failed (threshold=3) [.........................................XX.X....XXX..XX....] duration:
52.394891619s
FAIL (2025-09-06 12:42:54 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 52.394919583s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
5 out of 60 is failed (threshold=3) [.......................................XX.X.X.X.............] duration:
1.287877607s target:
https://us1.storj.io/login
FAIL (2025-09-06 12:42:52 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout
Duration: 1.287913625s
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
24 out of 60 is failed (threshold=3) [....................................XXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.000568141s
FAIL (2025-09-06 12:42:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000587729s
edge
gateway
us-select-1
|