Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 1m0.000710731s FAIL (2025-09-06 09:39:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00072123s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXX...XXXXX] duration: 30.00097225s FAIL (2025-09-06 09:39:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00099798s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 5 out of 60 is failed (threshold=3) [........................................X..X....X.X........X] duration: 30.178617811s FAIL (2025-09-06 09:39:00 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 47) Duration: 30.1786433s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration: 30.028030127s FAIL (2025-09-06 09:38:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.028052189s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [........................................X.X.X.X.XX.XXX.X..X.] duration: 19.484432564s FAIL (2025-09-06 09:38:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 19.484457632s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 6 out of 60 is failed (threshold=3) [...............................................XX...XX....XX] duration: 1m0.001122844s FAIL (2025-09-06 09:38:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:37:55.863105793 (187h29m52.543301362s) [upload=164988]: Starting upload Sep 6 09:37:55.863173643 (187h29m52.543369212s) [upload=164988]: Done starting upload Sep 6 09:37:55.863228648 (187h29m52.543424217s) [upload=164988]: Uploading segments... Sep 6 09:37:55.863279486 (187h29m52.543475055s) [upload=164988]: Got next segment. Inline: true Sep 6 09:37:55.863283854 (187h29m52.543479423s) [upload=164988]: Waiting for error group managing segments... Sep 6 09:37:55.863333008 (187h29m52.543528577s) [upload=164988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:37:55.863337176 (187h29m52.543532745s) [upload=164988]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:37:55.863339741 (187h29m52.5435353s) [upload=164988]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:38:01.601105739 (187h29m58.281301308s) [upload=164988]: Done uploading segments... Sep 6 09:38:01.601129996 (187h29m58.281325605s) [upload=164988]: Upload finished. err: <nil> Duration:* 1m0.001145668s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 3.596508908s FAIL (2025-09-06 09:38:47 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 46) Duration: 3.596536762s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 31.438932584s FAIL (2025-09-06 09:38:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== 77.90.31.15:4697 , piece_id=WTQRBNJMZPKVZ5ZVGQ3T6RI6D2NNG4KLQH4ETZOUWH6VKXMVTNBQ, seg_pos={0 0}, upload=164996]: Uploading piece... Sep 6 09:38:30.182632227 (187h30m26.862827807s) [noise=true , address=77.90.31.19:4842 , piece_id=OUZBIY5LGN24WXCTQ47Y2XN3FZDLC673RIYLV7TYW3PNKH7MVILQ, seg_pos={0 0}, upload=164996]: Uploading piece... Sep 6 09:38:30.183084903 (187h30m26.863280492s) [noise=true , address=45.91.116.4:4938 , piece_id=GWFZZOTM3HA4LQGFJAHFTL7VKA5CKT37NDXRLJXAQZQMSGFYZDPA, seg_pos={0 0}, upload=164996]: Uploading piece... Sep 6 09:38:30.183205474 (187h30m26.863401063s) [noise=true , address=45.91.116.10:4083 , piece_id=SSU4GAEJN5GCRO5BRP5N7NYQNMMFH3R57CJT6P75UBLQRD6WM6LQ, seg_pos={0 0}, upload=164996]: Uploading piece... Sep 6 09:38:30.183487844 (187h30m26.863683433s) [noise=true , address=45.91.116.9:4263 , piece_id=GSAB5BG5UT4BQY6XNQDZ36XMPHGXMVERO26M3UE3VW42TQBUYKAQ, seg_pos={0 0}, upload=164996]: Uploading piece... Sep 6 09:38:30.183843014 (187h30m26.864038583s) [noise=true , address=45.91.116.8:4047 , piece_id=QPHV27I6PWOK7YQPRTGEQCC77V3NOSF5M5SOUFD4IO54IWNCO6CQ, seg_pos={0 0}, upload=164996]: Uploading piece... Sep 6 09:38:30.184194928 (187h30m26.864390507s) [noise=true , address=45.91.116.13:4154 , piece_id=Q5UAN4D5EW6RUWHDQ75BHPIJ544GBCKZSFALXL7A44TAQKSKNUZQ, seg_pos={0 0}, upload=164996]: Uploading piece... Sep 6 09:38:30.184503818 (187h30m26.864699387s) [noise=true , address=45.91.116.10:4081 , piece_id=4EXAJHGRMLMCVNBOEGNVSLDG73UDRGIDQF75VE3A3R6IS2SCCBZA, seg_pos={0 0}, upload=164996]: Uploading piece... Sep 6 09:38:30.185659560 (187h30m26.865855129s) [noise=true , address=45.91.116.11:4117 , piece_id=QB6RLTSKX2B7BPMNCKGOOVTTVZKJ3EUFSX3VUZ5277G6JMFWJYOA, seg_pos={0 0}, upload=164996]: Uploading piece... Sep 6 09:38:44.948183280 (187h30m41.628378849s) [upload=164996]: Next segment err: context deadline exceeded Sep 6 09:38:44.948725247 (187h30m41.628920816s) [noise=true , address=45.91.116.6:4975 , piece_id=E52GKU2SYYDCFVKO4DHK543HWEUHOJP4EPL6JFZKYHQQ3BVB2N4Q, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.948883139 (187h30m41.629078718s) [noise=true , address=45.91.116.11:4118 , piece_id=Y7A42JDA37USH3O3OBVXBPEVIPODSCO2ORDKLYNSCBYLLATNF57A, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 188.166.225.170:54978->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:54978->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54978->45.91.116.11:4118: use of closed network connection Sep 6 09:38:44.948902466 (187h30m41.629098025s) [noise=true , address=77.90.31.16:4733 , piece_id=CAIFU4AULJXKFQVDD2UGUYJ4B5HLM5IWN3WBB4AOUXVWAGFDNHUA, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: read tcp 188.166.225.170:47314->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949181720 (187h30m41.629377299s) [noise=true , address=77.90.31.14:4663 , piece_id=JKYSPU7SXDMWNB2MYHQOLOMBK5SHNWMMAVSGVJIQLFI7XWHOCKNQ, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 188.166.225.170:50100->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949224242 (187h30m41.629419811s) [noise=true , address=45.91.116.4:4940 , piece_id=KIQPRQMP55PIYCUH4Q7VFEQGAK3D5QUPFSZ6TDCOULT5HX45IEIQ, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: read tcp 188.166.225.170:35466->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949399016 (187h30m41.629594585s) [noise=true , address=77.90.31.19:4842 , piece_id=OUZBIY5LGN24WXCTQ47Y2XN3FZDLC673RIYLV7TYW3PNKH7MVILQ, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:46338->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949446277 (187h30m41.629641846s) [noise=true , address=45.91.116.4:4938 , piece_id=GWFZZOTM3HA4LQGFJAHFTL7VKA5CKT37NDXRLJXAQZQMSGFYZDPA, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 188.166.225.170:42560->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949483087 (187h30m41.629678656s) [noise=true , address=77.90.31.16:4735 , piece_id=FWZFSMFCXITUNDFD27W2FA33MTVQPKGEWVCEAND3OV7UNKITIXDQ, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 188.166.225.170:41296->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949609649 (187h30m41.629805218s) [noise=true , address=45.91.116.10:4081 , piece_id=4EXAJHGRMLMCVNBOEGNVSLDG73UDRGIDQF75VE3A3R6IS2SCCBZA, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 188.166.225.170:35554->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949658553 (187h30m41.629854122s) [noise=true , address=77.90.31.15:4697 , piece_id=WTQRBNJMZPKVZ5ZVGQ3T6RI6D2NNG4KLQH4ETZOUWH6VKXMVTNBQ, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:38874->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:38874->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38874->77.90.31.15:4697: use of closed network connection Sep 6 09:38:44.949686446 (187h30m41.629882015s) [noise=true , address=45.91.116.10:4082 , piece_id=XKB23764MGO4EZBZTEZS64M7RHLFLQVKOZVJFC6OOK7K3BARG5PA, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 188.166.225.170:37474->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949720471 (187h30m41.62991604s) [noise=true , address=45.91.116.9:4263 , piece_id=GSAB5BG5UT4BQY6XNQDZ36XMPHGXMVERO26M3UE3VW42TQBUYKAQ, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 188.166.225.170:58562->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949759126 (187h30m41.629954695s) [noise=true , address=45.91.116.10:4083 , piece_id=SSU4GAEJN5GCRO5BRP5N7NYQNMMFH3R57CJT6P75UBLQRD6WM6LQ, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 188.166.225.170:47128->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949769726 (187h30m41.629965295s) [noise=true , address=45.91.116.8:4047 , piece_id=QPHV27I6PWOK7YQPRTGEQCC77V3NOSF5M5SOUFD4IO54IWNCO6CQ, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: read tcp 188.166.225.170:59160->45.91.116.8:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949787390 (187h30m41.629982959s) [noise=true , address=45.91.116.11:4117 , piece_id=QB6RLTSKX2B7BPMNCKGOOVTTVZKJ3EUFSX3VUZ5277G6JMFWJYOA, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 188.166.225.170:38876->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949840832 (187h30m41.630036401s) [noise=true , address=45.91.116.13:4154 , piece_id=Q5UAN4D5EW6RUWHDQ75BHPIJ544GBCKZSFALXL7A44TAQKSKNUZQ, seg_pos={0 0}, upload=164996]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 188.166.225.170:56694->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:44.949864036 (187h30m41.630059605s) [seg_pos={0 0}, upload=164996]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 31.438957562s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 7 out of 60 is failed (threshold=3) [..............................................XXX....X.XX..X] duration: 30.000404195s target: https://us1.storj.io/login FAIL (2025-09-06 09:38:34 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 30.000441006s core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 12 out of 60 is failed (threshold=3) [........................................XXXXXX.....XXX.X.X.X] duration: 30.000764112s FAIL (2025-09-06 09:38:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:38:13.656933432 (187h30m10.337129001s) [upload=164994]: Starting upload Sep 6 09:38:13.656994739 (187h30m10.337190308s) [upload=164994]: Done starting upload Sep 6 09:38:13.657048192 (187h30m10.337243761s) [upload=164994]: Uploading segments... Sep 6 09:38:13.657083198 (187h30m10.337278767s) [upload=164994]: Got next segment. Inline: true Sep 6 09:38:13.657087116 (187h30m10.337282685s) [upload=164994]: Waiting for error group managing segments... Sep 6 09:38:13.657122995 (187h30m10.337318564s) [upload=164994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:38:13.657127183 (187h30m10.337322752s) [upload=164994]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:38:13.657130509 (187h30m10.337326078s) [upload=164994]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:38:14.480166590 (187h30m11.160362159s) [upload=164994]: Done uploading segments... Sep 6 09:38:14.480186959 (187h30m11.160382528s) [upload=164994]: Upload finished. err: <nil> Duration:* 30.000786906s core uplink us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 866.204896ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-06 09:38:22 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 866.227179ms edge linksharing us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 30.000923383s FAIL (2025-09-06 09:38:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000936367s edge gateway us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 7.081448804s target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-06 09:38:10 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 7.081466056s edge linksharing us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 30.001232436s FAIL (2025-09-06 09:38:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Sep 6 09:38:05.450349106 (187h30m2.130544705s) [upload=164992]: Starting upload Sep 6 09:38:05.450458074 (187h30m2.130653643s) [upload=164992]: Done starting upload Sep 6 09:38:05.450566151 (187h30m2.13076173s) [upload=164992]: Uploading segments... Sep 6 09:38:05.450916402 (187h30m2.131111991s) [upload=164992]: Got next segment. Inline: false Sep 6 09:38:05.450974402 (187h30m2.131170012s) [upload=164992]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:38:05.450982568 (187h30m2.131178157s) [upload=164992]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 09:38:09.368322197 (187h30m6.048517766s) [upload=164992]: Next segment err: context deadline exceeded Sep 6 09:38:09.368397922 (187h30m6.048593492s) [upload=164992]: Done uploading segments... Sep 6 09:38:09.368414083 (187h30m6.048609652s) [upload=164992]: Upload finished. err: context deadline exceeded Duration: 30.001254809s core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.179253521s FAIL (2025-09-06 09:38:08 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 46) Duration: 5.179285473s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 1m4.360652245s FAIL (2025-09-06 09:38:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0) ======LOGS====== 0 , piece_id=IWGWW42INKQ7C2SXJ32GOKHEMXGLLAPRRSAYFWU6BJN73MTD5W3A, seg_pos={0 0}, upload=164984]: Uploading piece... Sep 6 09:37:21.620493115 (187h29m18.300688684s) [noise=true , address=145.79.227.32:4849 , piece_id=NTFEU44EONK5V5VQYFJLGRRNHHUUGREVUH5KR3Z7LRYQTJ4PF67Q, seg_pos={0 0}, upload=164984]: Uploading piece... Sep 6 09:37:21.620569491 (187h29m18.30076506s) [noise=true , address=145.79.227.6:4265 , piece_id=CBUVRMY7GFCLJEIUF5AHQZYD2DUNI7V3SEYCSLUQ5V3EO3HH6RFQ, seg_pos={0 0}, upload=164984]: Uploading piece... Sep 6 09:38:03.535601151 (187h30m0.21579673s) [upload=164984]: Next segment err: context deadline exceeded Sep 6 09:38:03.536625452 (187h30m0.216821021s) [noise=true , address=145.79.227.37:4575 , piece_id=KVOXMNAXZCQUNKN2WZV5X6KGLKEJCMVDULUHARL7BGXO546MTQ5Q, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:12faeDgwCPWMN4w1RxSPWwPTGNbaKnA87zYX8dhFRgm7F2YLYbV, address:145.79.227.37:4575): protocol: write tcp 188.166.225.170:54084->145.79.227.37:4575: use of closed network connection; write tcp 188.166.225.170:54084->145.79.227.37:4575: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54084->145.79.227.37:4575: use of closed network connection Sep 6 09:38:03.536878606 (187h30m0.217074175s) [noise=true , address=45.91.116.10:4113 , piece_id=JGUGOCOZIGTPTOWWI3GRYGY5RIS4ZQN3XNKFGUKP5A4IEQIHKJMA, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:12puRMny4nbgv2by4tAbXcBBSgG2d6BfwcDDRDcFzEdjSVUJxtd, address:45.91.116.10:4113): protocol: rpc: dial tcp 45.91.116.10:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.536943339 (187h30m0.217138908s) [noise=true , address=145.79.227.19:4098 , piece_id=4TYMLEHW66JCN242HEDQSBSP6PLNYPG2BB7KGL7C6SI34QI52LJA, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:122fmWY53WxM6KHuXzdKsCbWWCtCFDUGowvSqALWUdFnbYVJoSY, address:145.79.227.19:4098): protocol: write tcp 188.166.225.170:38484->145.79.227.19:4098: use of closed network connection; write tcp 188.166.225.170:38484->145.79.227.19:4098: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38484->145.79.227.19:4098: use of closed network connection Sep 6 09:38:03.537109649 (187h30m0.217305218s) [noise=true , address=145.79.227.14:4841 , piece_id=2OCI3RIKJLX464HY34BR7SPAKTU6ACPSBMVDECASAAJ6MJF66FZA, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:12ndNj9Dy8qQSbHXwVWw31sz19ki1qy6Zgetjvj1A5Q3odkzmDm, address:145.79.227.14:4841): protocol: write tcp 188.166.225.170:37302->145.79.227.14:4841: use of closed network connection; write tcp 188.166.225.170:37302->145.79.227.14:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37302->145.79.227.14:4841: use of closed network connection Sep 6 09:38:03.537136168 (187h30m0.217331737s) [noise=true , address=45.91.116.9:4283 , piece_id=I5D4EWWS6I3AKKG73DOB5IYVXNMSNOPAX3GVDZVIMNRP7AUMTS4A, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:1bquxzprFQQwaeW4KbSR5i8CBP6XCcHAWPmAmoszcsLUWKKT5g, address:45.91.116.9:4283): protocol: write tcp 188.166.225.170:51442->45.91.116.9:4283: use of closed network connection; write tcp 188.166.225.170:51442->45.91.116.9:4283: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->45.91.116.9:4283: use of closed network connection Sep 6 09:38:03.537215390 (187h30m0.217410959s) [noise=true , address=77.90.31.14:4665 , piece_id=TXYNIX7G5IWYUXDRBCSVAXDZHE3APTHSXHFN6U6C7FUDT5VNBSAA, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce, address:77.90.31.14:4665): protocol: read tcp 188.166.225.170:43778->77.90.31.14:4665: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537247051 (187h30m0.21744262s) [noise=true , address=45.91.116.12:4250 , piece_id=IWGWW42INKQ7C2SXJ32GOKHEMXGLLAPRRSAYFWU6BJN73MTD5W3A, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:1KvoMM3ptQXSxPhXFe3UoS145h8n3DMT9qmeaoR3bqNa8KLMm9, address:45.91.116.12:4250): protocol: read tcp 188.166.225.170:49420->45.91.116.12:4250: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537285555 (187h30m0.217481124s) [noise=true , address=145.79.227.6:4265 , piece_id=CBUVRMY7GFCLJEIUF5AHQZYD2DUNI7V3SEYCSLUQ5V3EO3HH6RFQ, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:1NscbwYiKarBdony9TwdccuEKg7f4VXEXerFfWmCiat4RKdXFi, address:145.79.227.6:4265): protocol: read tcp 188.166.225.170:46234->145.79.227.6:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537324308 (187h30m0.217519878s) [noise=true , address=145.79.227.13:4810 , piece_id=Q4LA3WQWABU3WJMX36JPDEWOINZ5EOCBW4OOW2HTAMK6YLNIUGHA, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:12GTWz2fWHLAApibeARUerDvJ9c4AWYRVAvumRxgJwwqCYYJJfg, address:145.79.227.13:4810): protocol: read tcp 188.166.225.170:32794->145.79.227.13:4810: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537337414 (187h30m0.217532983s) [noise=true , address=145.79.227.36:4536 , piece_id=H5NFMZ5ZN2NUL3WMB3O2F227PRY3HYPEMO4KHHGORLMLGC3XCEOQ, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:1WoVNAp5RfVviaxN4RWCUDEhxLKsdGB8bgE6gcUcgZKcM4Cz1s, address:145.79.227.36:4536): protocol: read tcp 188.166.225.170:56376->145.79.227.36:4536: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537381317 (187h30m0.217576887s) [noise=true , address=77.90.31.10:4521 , piece_id=DXDU76S55IDCUOOV7643JD5UBM456MIJ4MXLOYUHK657LAJFAWDQ, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:1WpVJa7SXkANKParDWYrC1cN8aLHTkUSPrRJGKQ6fvA5oDEkoa, address:77.90.31.10:4521): protocol: read tcp 188.166.225.170:47172->77.90.31.10:4521: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537392650 (187h30m0.217588219s) [noise=true , address=145.79.227.44:4201 , piece_id=MDUZQ4QG5KHOCVWJZTN6KVAW3XBSULE2KBSXTGVRASBNVPY5RFIQ, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi, address:145.79.227.44:4201): protocol: read tcp 188.166.225.170:58946->145.79.227.44:4201: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537417698 (187h30m0.217613267s) [noise=true , address=145.79.227.35:4483 , piece_id=HUHIB6ZOKGIAVEJNBMTZUCKKQ7R3NLC54NVZGY7RVZJBJA6C4QVA, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:1igSs2ZGX3zWVsSdmm1pnwYd7JckcyBubTuNU1gJDxQsZ9nKaY, address:145.79.227.35:4483): protocol: read tcp 188.166.225.170:49046->145.79.227.35:4483: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537434930 (187h30m0.217630499s) [noise=true , address=145.79.227.32:4849 , piece_id=NTFEU44EONK5V5VQYFJLGRRNHHUUGREVUH5KR3Z7LRYQTJ4PF67Q, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:12goRcYW39Qqga4HW1i6cg6VGKJFUxnyTJBe6HRnZiLAU3LonWn, address:145.79.227.32:4849): protocol: read tcp 188.166.225.170:43220->145.79.227.32:4849: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537459768 (187h30m0.217655327s) [noise=true , address=145.79.227.17:4001 , piece_id=MICCZWJFZLAYRTETZV2MO35S4RS3IRGQUSTXMJM7PY2BUZUT5H2A, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:12Eaef5RrMHiZS6dE7wTXyq239F1RXovU7UCTQ7uC6Qusj6bJDw, address:145.79.227.17:4001): protocol: read tcp 188.166.225.170:53822->145.79.227.17:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537487360 (187h30m0.21768293s) [noise=true , address=145.79.227.33:4883 , piece_id=TT5ULTD3JUN6DGAZNMJYA3BSJP5JTBTFQWHSOYBXF6RYQBLKMFVA, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:12cx1avL4umDm2pzJS1rNZqeDZjZMWnMtNsyNTdMdkxhpa1ByUN, address:145.79.227.33:4883): protocol: read tcp 188.166.225.170:39000->145.79.227.33:4883: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537492411 (187h30m0.21768798s) [noise=true , address=45.91.116.6:4990 , piece_id=S3653ZUZGJ6UUSGPMPOXATDMEHA6PCIRIEUUXYQWNAC3RZ2HJ6EA, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:126JXfJvuad1JedXjgEJuA12awNtwHBYNZhHSJoei2ZYTrccDmA, address:45.91.116.6:4990): protocol: read tcp 188.166.225.170:38954->45.91.116.6:4990: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537547807 (187h30m0.217743376s) [noise=true , address=77.90.31.17:4785 , piece_id=VZLYK3RQX7TMZMTD5BRPFKM4BYIURGFU2BZNEX66Z67S37KJLK4Q, seg_pos={0 0}, upload=164984]: Done uploading piece. err: ecclient: upload failed (node:1P9AKP1rxpVvkxDs6XH2S6hR8DRHBZkWMZiYPTcZZjMZwLAaBN, address:77.90.31.17:4785): protocol: read tcp 188.166.225.170:47640->77.90.31.17:4785: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:03.537568997 (187h30m0.217764566s) [seg_pos={0 0}, upload=164984]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil> Duration: 1m4.36066589s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 1m0.178068481s FAIL (2025-09-06 09:38:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 7) ======LOGS====== 7GJKMZSXDQXJ3D2ZAHILTNLZMV6PRA, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:59546->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:59546->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59546->81.171.14.70:4045: use of closed network connection Sep 6 09:38:07.007020336 (187h30m3.687215905s) [noise=true , address=81.171.14.70:4046 , piece_id=XMZVSZCUCBZF4KOGV3PDYQYGQUB6QXTGCPTDQE6BUEAW2J2MKRCA, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:51404->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:07.007059081 (187h30m3.68725464s) [noise=true , address=185.167.204.206:4053 , piece_id=HSMEVXGON2E55PAYAPWBDCWUIMBROPMOTI7WTVEQPXNH4JG3E4LQ, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 188.166.225.170:54262->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:07.007089529 (187h30m3.687285088s) [noise=true , address=185.167.204.205:4016 , piece_id=2YJI72NRHX2QYVWHWZIIMLD6WM6EKC72IKXKDRP6JD3H3S7TP2WQ, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: read tcp 188.166.225.170:36484->185.167.204.205:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:07.007120929 (187h30m3.687316498s) [noise=true , address=185.167.204.205:4018 , piece_id=SZAO6BF2Z2Y2TPY5YZFT7CLMKKR3QOXUBUPOZXKG6NSNI3WOPR5A, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:38486->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:07.007156827 (187h30m3.687352396s) [noise=true , address=81.171.14.70:4030 , piece_id=NYQHJX7MJNWLN5VVDKXZOES6DACKUEY7SDIMFDANBPY2UPCQW6CQ, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:33542->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:33542->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33542->81.171.14.70:4030: use of closed network connection Sep 6 09:38:07.007244205 (187h30m3.687439774s) [noise=true , address=185.167.204.206:4057 , piece_id=SADPHRZC5UX2J2Z4PW5XHVVSBBZH35MPF6RQJGWNRXIONQ3LUTBA, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:38:07.007295372 (187h30m3.687490941s) [noise=true , address=185.167.204.205:4022 , piece_id=7LGP6VHXI7PFAC67WTMYUHMDEEGCWQF5RAI54VZ75J5FCYXQFCNQ, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: write tcp 188.166.225.170:34456->185.167.204.205:4022: use of closed network connection; write tcp 188.166.225.170:34456->185.167.204.205:4022: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34456->185.167.204.205:4022: use of closed network connection Sep 6 09:38:07.007340309 (187h30m3.687535878s) [noise=true , address=185.167.204.205:4011 , piece_id=SZANKYY5MHXBBEHN5JFXBCQLNREU5HA5EF5LSYKA6GECBQOGE2MQ, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: write tcp 188.166.225.170:38754->185.167.204.205:4011: use of closed network connection; write tcp 188.166.225.170:38754->185.167.204.205:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38754->185.167.204.205:4011: use of closed network connection Sep 6 09:38:07.007378311 (187h30m3.68757389s) [noise=true , address=81.171.14.70:4032 , piece_id=CV5QRNIEEGESINUJ23J55AXOQMXV7M7LKBAXGUIBOYSF3GPGRUSA, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:40154->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:40154->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40154->81.171.14.70:4032: use of closed network connection Sep 6 09:38:07.007484705 (187h30m3.687680284s) [noise=true , address=81.171.14.70:4038 , piece_id=4OXKR2IAWH4VV34EBVNQ3U2SHUNPHZXXSLEMQXXSRZQRIEVJSIYA, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:54210->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:54210->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->81.171.14.70:4038: use of closed network connection Sep 6 09:38:07.007571120 (187h30m3.687766689s) [noise=true , address=81.171.14.70:4044 , piece_id=LDDZFHS6A7AYBKIUGAZGOQP4GE4LQDIL2BR2BKCM34SN3TOYTPFQ, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: read tcp 188.166.225.170:47366->81.171.14.70:4044: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:07.007585868 (187h30m3.687781438s) [noise=true , address=185.167.204.205:4027 , piece_id=IIKJ3HXRUQGRAMC77TTN2QPYABYV22XOU5KFINZJXLERJE4QGU4Q, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: read tcp 188.166.225.170:52600->185.167.204.205:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:07.007606218 (187h30m3.687801787s) [noise=true , address=185.167.204.205:4004 , piece_id=VTVKRNP62O2PQUCOPBPYCE6DQP5WLU4OKCSX7UZ37RED563LEBSQ, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 188.166.225.170:56922->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:07.007635313 (187h30m3.687830872s) [noise=true , address=185.167.204.206:4063 , piece_id=5RZVJ4JHC4OJB67JVEZDLZQNW2TWAUNER33WJJDCYISIQNGP65MA, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:07.007664709 (187h30m3.687860279s) [noise=true , address=81.171.14.70:4037 , piece_id=IMZ6Q5BGYMJBSYRYKOFDLPJIXY3KCOGFFKRPJ3YJPM53UZZF5BNA, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:40932->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:40932->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40932->81.171.14.70:4037: use of closed network connection Sep 6 09:38:07.007709075 (187h30m3.687904644s) [noise=true , address=185.167.204.205:4020 , piece_id=JOM5IKZZCY4H4YGSYSCNLAH4AGTT74J3KYMI4A4JNKBPNJ56VOCQ, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: read tcp 188.166.225.170:53380->185.167.204.205:4020: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:38:07.007747768 (187h30m3.687943337s) [noise=true , address=185.167.204.205:4003 , piece_id=LZMZBKSUWHXZ6UXTZI6FPF5KGRWX6JIT5PJNGDJZKIYKLCY62XJA, seg_pos={0 0}, upload=164983]: Done uploading piece. err: ecclient: upload failed (node:1CjWvvJ4FdZc41sRgsQFWz4Y1ATZKgzsvDcUTmBSrCC15ekCmF, address:185.167.204.205:4003): protocol: write tcp 188.166.225.170:56520->185.167.204.205:4003: use of closed network connection; write tcp 188.166.225.170:56520->185.167.204.205:4003: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56520->185.167.204.205:4003: use of closed network connection Sep 6 09:38:07.007788566 (187h30m3.687984135s) [seg_pos={0 0}, upload=164983]: Done waiting for segment. successful: 7 optimal: 26 errs: <nil> Sep 6 09:38:07.182439047 (187h30m3.862634616s) [upload=164983]: Done uploading segments... Sep 6 09:38:07.182468183 (187h30m3.862663752s) [upload=164983]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 7) Duration: 1m0.178080674s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 1m0.001267053s FAIL (2025-09-06 09:38:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001304545s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 16 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXX...XXXX] duration: 30.000585981s FAIL (2025-09-06 09:38:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000602634s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 5 out of 60 is failed (threshold=3) [...............................................X..X..X..X.X.] duration: 3.581239291s FAIL (2025-09-06 09:38:04 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 3.581260922s edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.........................................X.X.X.X.XX.XXX.X..X] duration: 30.001218324s FAIL (2025-09-06 09:38:03 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001244323s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 34.301048327s FAIL (2025-09-06 09:37:55 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 48) Duration: 34.301072594s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 30.000303922s FAIL (2025-09-06 09:37:43 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 30.000332286s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 5 out of 60 is failed (threshold=3) [................................................XX...XX....X] duration: 1m0.001735695s FAIL (2025-09-06 09:37:41 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:36:57.620684710 (187h28m54.300880279s) [upload=164977]: Starting upload Sep 6 09:36:57.620750926 (187h28m54.300946495s) [upload=164977]: Done starting upload Sep 6 09:36:57.620821341 (187h28m54.30101691s) [upload=164977]: Uploading segments... Sep 6 09:36:57.620867148 (187h28m54.301062717s) [upload=164977]: Got next segment. Inline: true Sep 6 09:36:57.620872349 (187h28m54.301067918s) [upload=164977]: Waiting for error group managing segments... Sep 6 09:36:57.620904160 (187h28m54.301099729s) [upload=164977]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:36:57.620908077 (187h28m54.301103636s) [upload=164977]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:36:57.620910431 (187h28m54.301106s) [upload=164977]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:37:16.603471920 (187h29m13.283667489s) [upload=164977]: Done uploading segments... Sep 6 09:37:16.603491017 (187h29m13.283686576s) [upload=164977]: Upload finished. err: <nil> Duration:* 1m0.001756645s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................................X..X....X.X........] duration: 5.362361268s FAIL (2025-09-06 09:37:38 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 47) Duration: 5.362387838s core uplink eu1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 6 out of 60 is failed (threshold=3) [...............................................XXX....X.XX..] duration: 24.622375923s remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login FAIL (2025-09-06 09:37:34 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 24.622395391s core satellite us1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 5 out of 60 is failed (threshold=3) [................................................X..X..X..X.X] duration: 30.001018056s FAIL (2025-09-06 09:37:34 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 30.001066198s edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 15 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXX...XXX] duration: 30.001380208s FAIL (2025-09-06 09:37:25 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001388633s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................X.X..X.] duration: 16.655473486s target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-06 09:37:20 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 16.65549703s edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 38.619172484s FAIL (2025-09-06 09:37:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== BBVWPUTXGMSKBMR6B2EOJ7E4TMOFCPP5UGWCWJWG6A, seg_pos={0 0}, upload=164976]: Uploading piece... Sep 6 09:36:50.955473471 (187h28m47.63566904s) [noise=true , address=45.91.116.11:4117 , piece_id=AUOLAAOMAWXUAAZUXWINVLNOS5ZJ567QXVB6ZYHF6EGLIYZZBRNA, seg_pos={0 0}, upload=164976]: Uploading piece... Sep 6 09:36:50.955722818 (187h28m47.635918487s) [noise=true , address=77.90.31.15:4698 , piece_id=VA6OFSU2G5MLI2UC6YAQEHHX5W5GRB3BUDH7TXNAP4VJJZ7S5R7A, seg_pos={0 0}, upload=164976]: Uploading piece... Sep 6 09:36:50.955762625 (187h28m47.635958194s) [noise=true , address=45.91.116.10:4083 , piece_id=PL43GW54FI3S4ZI4OCZ533OBKCZ3JYNAJZET4INY3R224OCHSOBQ, seg_pos={0 0}, upload=164976]: Uploading piece... Sep 6 09:36:50.956082226 (187h28m47.636277795s) [noise=true , address=45.91.116.4:4940 , piece_id=LZASSYD4SOB6YMFD6OY65HADRNSJGHADGDBQY3HAUJVKBKC33BRQ, seg_pos={0 0}, upload=164976]: Uploading piece... Sep 6 09:36:50.956304030 (187h28m47.636499599s) [noise=true , address=77.90.31.13:4626 , piece_id=F2HH3QBZH4BBDYBO5FICPBTFRJ7VSGRK2CFPSYUC5GA4GXHQE7CQ, seg_pos={0 0}, upload=164976]: Uploading piece... Sep 6 09:36:50.956320662 (187h28m47.636516231s) [noise=true , address=45.91.116.8:4047 , piece_id=KRS675QLPP3PB4UB74LMCKY2BTM46JKDYAAKKFIEFFQDMDNWNOLA, seg_pos={0 0}, upload=164976]: Uploading piece... Sep 6 09:36:50.956559820 (187h28m47.636755389s) [noise=true , address=77.90.31.13:4628 , piece_id=KG5BX33WCGIFE62ZMCIXDENOHRDKTKXQCL62YD7Y3XYL4XHQJRRQ, seg_pos={0 0}, upload=164976]: Uploading piece... Sep 6 09:36:50.956808125 (187h28m47.637003694s) [noise=true , address=45.91.116.13:4154 , piece_id=ISMSI32RL3Q45AUR4CNXHUXXLWP6ZRX62WJ6D6N4L25IQFHZEK4A, seg_pos={0 0}, upload=164976]: Uploading piece... Sep 6 09:36:50.957030601 (187h28m47.63722616s) [noise=true , address=77.90.31.14:4663 , piece_id=LHXZ7QN2WGSPMDRG3NNRMLISQL3PY6GSH5B3JDI4TZMIKMNCJ5EA, seg_pos={0 0}, upload=164976]: Uploading piece... Sep 6 09:37:08.205142943 (187h29m4.885338512s) [upload=164976]: Next segment err: context deadline exceeded Sep 6 09:37:08.205829827 (187h29m4.886025396s) [noise=true , address=45.91.116.10:4083 , piece_id=PL43GW54FI3S4ZI4OCZ533OBKCZ3JYNAJZET4INY3R224OCHSOBQ, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: rpc: dial tcp 45.91.116.10:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206346155 (187h29m4.886541725s) [noise=true , address=77.90.31.15:4699 , piece_id=VI52WSEBSMDCTNLLRBTYNW6ET5M3OG447Y7FAGSBE3PZ2X3WQDHQ, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:41372->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:41372->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41372->77.90.31.15:4699: use of closed network connection Sep 6 09:37:08.206603778 (187h29m4.886799347s) [noise=true , address=45.91.116.4:4940 , piece_id=LZASSYD4SOB6YMFD6OY65HADRNSJGHADGDBQY3HAUJVKBKC33BRQ, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: read tcp 188.166.225.170:48794->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206661268 (187h29m4.886856837s) [noise=true , address=77.90.31.14:4663 , piece_id=LHXZ7QN2WGSPMDRG3NNRMLISQL3PY6GSH5B3JDI4TZMIKMNCJ5EA, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 188.166.225.170:38468->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206693570 (187h29m4.886889129s) [noise=true , address=77.90.31.10:4520 , piece_id=Y6FL76F54RIYQ757GCK2RMOP6ALQG4D64GO6WECF5NN7V6IJHDGQ, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 188.166.225.170:46756->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206722956 (187h29m4.886918525s) [noise=true , address=77.90.31.13:4628 , piece_id=KG5BX33WCGIFE62ZMCIXDENOHRDKTKXQCL62YD7Y3XYL4XHQJRRQ, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:43036->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206751161 (187h29m4.88694673s) [noise=true , address=45.91.116.13:4154 , piece_id=ISMSI32RL3Q45AUR4CNXHUXXLWP6ZRX62WJ6D6N4L25IQFHZEK4A, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 188.166.225.170:46508->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206780677 (187h29m4.886976246s) [noise=true , address=45.91.116.8:4048 , piece_id=B3LM6J5ATW3V3LDFD5F5BLJWLYECFXWGZKE2ZOMA4ZS7FIC2ZI7A, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 188.166.225.170:60546->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206812578 (187h29m4.887008147s) [noise=true , address=77.90.31.15:4698 , piece_id=VA6OFSU2G5MLI2UC6YAQEHHX5W5GRB3BUDH7TXNAP4VJJZ7S5R7A, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: read tcp 188.166.225.170:43810->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206871160 (187h29m4.887066719s) [noise=true , address=77.90.31.13:4626 , piece_id=F2HH3QBZH4BBDYBO5FICPBTFRJ7VSGRK2CFPSYUC5GA4GXHQE7CQ, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 188.166.225.170:36556->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206899765 (187h29m4.887095334s) [noise=true , address=77.90.31.19:4842 , piece_id=LOXOOTI6UTJGWU3YX5WUISDOECUYK6SGDE7A5JX2MOXL4TD6IJQA, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:45848->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206930674 (187h29m4.887126243s) [noise=true , address=77.90.31.16:4735 , piece_id=R3TUKSVKXAUWNBELD4UC2WG23P47X3X75IQIHZXUE7PTUXHV4EAA, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 188.166.225.170:59248->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206957084 (187h29m4.887152653s) [noise=true , address=45.91.116.9:4263 , piece_id=HCHQHSANJSBBVWPUTXGMSKBMR6B2EOJ7E4TMOFCPP5UGWCWJWG6A, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 188.166.225.170:45630->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.206985859 (187h29m4.887181428s) [noise=true , address=45.91.116.8:4047 , piece_id=KRS675QLPP3PB4UB74LMCKY2BTM46JKDYAAKKFIEFFQDMDNWNOLA, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: read tcp 188.166.225.170:45210->45.91.116.8:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.207014905 (187h29m4.887210464s) [noise=true , address=45.91.116.11:4119 , piece_id=27OBSRXHJ5NMYSAZLNKPNQZODMZ7K46ZL4ZDQVNGNQZDYKU77CPQ, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 188.166.225.170:35586->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.207040864 (187h29m4.887236433s) [noise=true , address=45.91.116.11:4117 , piece_id=AUOLAAOMAWXUAAZUXWINVLNOS5ZJ567QXVB6ZYHF6EGLIYZZBRNA, seg_pos={0 0}, upload=164976]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 188.166.225.170:36956->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:08.207053429 (187h29m4.887248998s) [seg_pos={0 0}, upload=164976]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 38.61919759s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [..........................................X.X.X.X.XX.XXX.X..] duration: 26.389397455s FAIL (2025-09-06 09:37:15 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 26.389428354s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................................X..X....X.X.......] duration: 5.168526588s FAIL (2025-09-06 09:37:12 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 47) Duration: 5.168545925s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.........................................XXXXXX.....XXX.X.X.] duration: 21.121282585s FAIL (2025-09-06 09:37:11 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:36:25.642132301 (187h28m22.322327871s) [upload=164973]: Starting upload Sep 6 09:36:25.642237984 (187h28m22.322433553s) [upload=164973]: Done starting upload Sep 6 09:36:25.642290845 (187h28m22.322486414s) [upload=164973]: Uploading segments... Sep 6 09:36:25.642329599 (187h28m22.322525168s) [upload=164973]: Got next segment. Inline: true Sep 6 09:36:25.642333516 (187h28m22.322529085s) [upload=164973]: Waiting for error group managing segments... Sep 6 09:36:25.642368694 (187h28m22.322564263s) [upload=164973]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:36:25.642373233 (187h28m22.322568802s) [upload=164973]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:36:25.642376078 (187h28m22.322571647s) [upload=164973]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:36:36.637846177 (187h28m33.318041746s) [upload=164973]: Done uploading segments... Sep 6 09:36:36.637867978 (187h28m33.318063547s) [upload=164973]: Upload finished. err: <nil> Duration:* 21.12130113s core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 3.544946833s FAIL (2025-09-06 09:37:07 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 46) Duration: 3.544974516s core uplink ap1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 1m0.172068794s FAIL (2025-09-06 09:37:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 6) ======LOGS====== 33: use of closed network connection Sep 6 09:37:06.588125526 (187h29m3.268321085s) [noise=true , address=185.167.204.205:4021 , piece_id=IWXEHC7SSHOEEUJF3YA65AG5EVSL37ATINMHIHMO7W6Y4TPFJTWA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: write tcp 188.166.225.170:50228->185.167.204.205:4021: use of closed network connection; write tcp 188.166.225.170:50228->185.167.204.205:4021: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50228->185.167.204.205:4021: use of closed network connection Sep 6 09:37:06.588212773 (187h29m3.268408342s) [noise=true , address=81.171.14.70:4036 , piece_id=RMNSU42R7JZMI2LZYZSCNW5VSBINHTXLHYVNOSR5P2BMH6JN762A, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:37004->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.588462821 (187h29m3.26865838s) [noise=true , address=81.171.14.70:4027 , piece_id=PK7YHSZRG5ZUGE3NFYRQTUEORUWGMTYNMYQBPCLUCLOIXGBOSUTA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: write tcp 188.166.225.170:36662->81.171.14.70:4027: use of closed network connection; write tcp 188.166.225.170:36662->81.171.14.70:4027: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36662->81.171.14.70:4027: use of closed network connection Sep 6 09:37:06.588521113 (187h29m3.268716682s) [noise=true , address=81.171.14.70:4048 , piece_id=Z62WKRT7A2XWG7EGGIM2M2H7S2KKRRKY7FK665LKG7BOIU2PGGMA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:43246->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:43246->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43246->81.171.14.70:4048: use of closed network connection Sep 6 09:37:06.588660730 (187h29m3.268856299s) [noise=true , address=185.167.204.206:4040 , piece_id=Z4PO73HYMQQXCROUDH7CON3FRE674NDYKNXZEJA3JJ4P2TVQVLRQ, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: write tcp 188.166.225.170:45196->185.167.204.206:4040: use of closed network connection; write tcp 188.166.225.170:45196->185.167.204.206:4040: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45196->185.167.204.206:4040: use of closed network connection Sep 6 09:37:06.588743849 (187h29m3.268939418s) [noise=true , address=81.171.14.70:4046 , piece_id=QHBD2OD7BHLR324OYTOSEHEPWFYG3EHEKWFIMDNJZPN6KINXHNMA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:33104->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:33104->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33104->81.171.14.70:4046: use of closed network connection Sep 6 09:37:06.588777514 (187h29m3.268973083s) [noise=true , address=81.171.14.70:4045 , piece_id=IW3WAXXZW7TA2QXNDXEJBRLM2K6AJMXLZIBXY6B4GNKUNKZA53OA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:38144->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:38144->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38144->81.171.14.70:4045: use of closed network connection Sep 6 09:37:06.588809705 (187h29m3.269005274s) [noise=true , address=81.171.14.70:4044 , piece_id=PHAHCYCCEINB56CR4KTGBMUWDNFKXBQYQ7GWJ5XES2L4VN6IZUMQ, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: read tcp 188.166.225.170:53582->81.171.14.70:4044: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.588853869 (187h29m3.269049438s) [noise=true , address=185.167.204.206:4041 , piece_id=NIYH4FKGECCFV5SYRIOXCIUGKKPUH7S7ASXOFCJ6Y55ZNLLOOSQQ, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: read tcp 188.166.225.170:48408->185.167.204.206:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.588882083 (187h29m3.269077652s) [noise=true , address=81.171.14.70:4042 , piece_id=3QWJ6Y6ZYTLVYIMAFGIS2MNL7U3FT65EYINLBHEIDPJQDZYZU3KA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:49712->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.588909275 (187h29m3.269104834s) [noise=true , address=185.167.204.205:4030 , piece_id=ROQ66LBSZWY5QXYQEFE4K5S4LTHNPWYLAHXJQNSMSJVWXABRVFDQ, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 188.166.225.170:49788->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.588935576 (187h29m3.269131145s) [noise=true , address=81.171.14.70:4033 , piece_id=H7VYMCTO5INHJJXBUIWIBPOMXSTAWXEWSUGDSDZ67GFW3SAOMSQQ, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50976->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.588963910 (187h29m3.269159479s) [noise=true , address=185.167.204.206:4063 , piece_id=FSRVLMJ5LZLSRJCWFG5XQVR3UCDHSMGMG5D7LEULZNTFVDLFV5OA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: read tcp 188.166.225.170:42372->185.167.204.206:4063: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.588990811 (187h29m3.26918638s) [noise=true , address=185.167.204.206:4071 , piece_id=3TXBTOJ5OE5ZEKRUNANSAGYIYBBSVWR2LH4W2V75YIDNSRAB7DRA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: read tcp 188.166.225.170:43228->185.167.204.206:4071: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.589031580 (187h29m3.269227149s) [noise=true , address=81.171.14.70:4028 , piece_id=WSL5EGUCPUHS4MWW5O2E7IVEZ5U3A266AWKN253TXOOCNV7O6DGQ, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: read tcp 188.166.225.170:57684->81.171.14.70:4028: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.589067538 (187h29m3.269263097s) [noise=true , address=185.167.204.205:4032 , piece_id=JXNX2CQFOIKX7XVMO577D7WBTRQVIOL3GAMM5BMGCBBHR7PATPCA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:12558dx7zMNDfabV55vSsP9XwdxXgoPM8aHW53femVZzkCRxsKN, address:185.167.204.205:4032): protocol: read tcp 188.166.225.170:54332->185.167.204.205:4032: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.589100842 (187h29m3.269296411s) [noise=true , address=81.171.14.70:4026 , piece_id=CCQ5GR5UX2DUKIINUAFP6DZV42GAKJC3HAZVLKG4RWKAGY43PIGA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:33654->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.589136451 (187h29m3.26933202s) [noise=true , address=185.167.204.206:4051 , piece_id=Q3MV3WLPWCPQQT7DYEHKDA26GFJUN6GE2ER4C55FSKDG22CNW5HA, seg_pos={0 0}, upload=164967]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: read tcp 188.166.225.170:50582->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:06.589150737 (187h29m3.269346296s) [seg_pos={0 0}, upload=164967]: Done waiting for segment. successful: 6 optimal: 26 errs: <nil> Sep 6 09:37:06.757080288 (187h29m3.437275867s) [upload=164967]: Done uploading segments... Sep 6 09:37:06.757106748 (187h29m3.437302317s) [upload=164967]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 6) Duration: 1m0.172086409s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 1m0.000712176s FAIL (2025-09-06 09:37:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000723608s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 1m1.434711241s FAIL (2025-09-06 09:37:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0) ======LOGS====== k connection; write tcp 188.166.225.170:51920->145.79.227.6:4289: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51920->145.79.227.6:4289: use of closed network connection Sep 6 09:37:01.861031729 (187h28m58.541227308s) [noise=true , address=145.79.227.4:4193 , piece_id=LI6Z7PL3U6XEH57FRK3OIDXYX2QRL6OR7NRIRE7NNHX4SXZF2HKQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:15XNLgptmXprcfkVMJsuzY2WDyHiZDpLetiBpmjXhb4c9q7yMd, address:145.79.227.4:4193): protocol: write tcp 188.166.225.170:42668->145.79.227.4:4193: use of closed network connection; write tcp 188.166.225.170:42668->145.79.227.4:4193: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42668->145.79.227.4:4193: use of closed network connection Sep 6 09:37:01.861204339 (187h28m58.541399908s) [noise=true , address=145.79.227.30:4780 , piece_id=SYLTP7FR26NA6BNE4DI326IQKADMF4AYMDV3ZKAEW57OLBJDQIOQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:125WCbZ6a2NaewiLAHhHuPo91sQEfC9CZzEBV6rwF1JY7z13AEY, address:145.79.227.30:4780): protocol: write tcp 188.166.225.170:46906->145.79.227.30:4780: use of closed network connection; write tcp 188.166.225.170:46906->145.79.227.30:4780: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46906->145.79.227.30:4780: use of closed network connection Sep 6 09:37:01.861394012 (187h28m58.541589581s) [noise=true , address=145.79.227.49:4357 , piece_id=D3ZSVWM75KKTQS2Q5RGYCYQECWRG5PTR7PBJREVD24OBARGZY6JQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi, address:145.79.227.49:4357): protocol: read tcp 188.166.225.170:59814->145.79.227.49:4357: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:01.861430542 (187h28m58.541626112s) [noise=true , address=145.79.227.52:4449 , piece_id=ZCOYPKWHLJEVUYVIJLF6W7O72ZXGOSEUTNWQLKPIFQNYGFPRCQPQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:12f14pHDEEamEnHmeZCe38rjaE6EYWYkGMPrZMsbSDmApN5rckq, address:145.79.227.52:4449): protocol: read tcp 188.166.225.170:60250->145.79.227.52:4449: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:01.861466701 (187h28m58.54166227s) [noise=true , address=145.79.227.40:4076 , piece_id=2BKXRGTPOFINKWCU67LYKBTX5IXE2ISBNH7ZG6V7BDJNZGVA52IQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:19KofWeQV4jMQzmSCv3LUsUqAqGfAuDqyo9GGsqKGi7ctT3FA2, address:145.79.227.40:4076): protocol: write tcp 188.166.225.170:58718->145.79.227.40:4076: use of closed network connection; write tcp 188.166.225.170:58718->145.79.227.40:4076: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58718->145.79.227.40:4076: use of closed network connection Sep 6 09:37:01.861621027 (187h28m58.541816606s) [noise=true , address=145.79.227.5:4234 , piece_id=D6I3OH563S6AVVXDMKAK5QOMABU4VIROMR7S66LMYVB2DEEPTPRQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:12DtCsEnAhXZnGRXjnhNfqNz4RvPnvTMDz1ur7DyZiQtQ7nbwDS, address:145.79.227.5:4234): protocol: write tcp 188.166.225.170:56490->145.79.227.5:4234: use of closed network connection; write tcp 188.166.225.170:56490->145.79.227.5:4234: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56490->145.79.227.5:4234: use of closed network connection Sep 6 09:37:01.861645363 (187h28m58.541840942s) [noise=true , address=145.79.227.27:4703 , piece_id=UB5LBDIS5T4NEU6CJKDZV2SUUTI2QCVHOHUMY2RB5ZJ7PPRWMLVQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:12nHGPE4LDH4wwTXZXA8vHqn4Tm735RvfC7mKx1jEMBYNbA1jNd, address:145.79.227.27:4703): protocol: write tcp 188.166.225.170:34130->145.79.227.27:4703: use of closed network connection; write tcp 188.166.225.170:34130->145.79.227.27:4703: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34130->145.79.227.27:4703: use of closed network connection Sep 6 09:37:01.861719144 (187h28m58.541914723s) [noise=true , address=77.90.31.17:4779 , piece_id=L6FFJCD2LD22FVIYAIZGSJWMIZY5ZO2MWZOE5WF6NQDUTBR4ZNMQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:12HfiiWLGzNiY2CiGyBUWZ11t1sRY4duPLvAMdfgi4jMetx2ePN, address:77.90.31.17:4779): protocol: write tcp 188.166.225.170:57256->77.90.31.17:4779: use of closed network connection; write tcp 188.166.225.170:57256->77.90.31.17:4779: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57256->77.90.31.17:4779: use of closed network connection Sep 6 09:37:01.861849163 (187h28m58.542044732s) [noise=true , address=145.79.227.14:4856 , piece_id=GIWEODKQ4A62M22MUWRWPGYDFITENL54XICTAOJZPITMIJPRLYUA, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:12mXTGa9rxx8jU8oKLmR9obX3Utn7p5rQH2GeK6LeJ6jiNvfYZ6, address:145.79.227.14:4856): protocol: write tcp 188.166.225.170:36142->145.79.227.14:4856: use of closed network connection; write tcp 188.166.225.170:36142->145.79.227.14:4856: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36142->145.79.227.14:4856: use of closed network connection Sep 6 09:37:01.861919739 (187h28m58.542115308s) [noise=true , address=145.79.227.39:4044 , piece_id=3U4GCU7PONSNL5ZZICVJYVVG6SVMJY5BBGP2FTK26WXFFBWTYU7A, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:1oEaCgnsStJePrKHSXLLmwsAuYPEmgFbejFAMmNMqLp8m8vxiC, address:145.79.227.39:4044): protocol: write tcp 188.166.225.170:52748->145.79.227.39:4044: use of closed network connection; write tcp 188.166.225.170:52748->145.79.227.39:4044: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52748->145.79.227.39:4044: use of closed network connection Sep 6 09:37:01.862092439 (187h28m58.542288008s) [noise=true , address=145.79.227.20:4448 , piece_id=TBZEMHUY63NDUVONZBG5FRGV4Z7AZ2HOVISSFQFFDZ77BXZXWBHA, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:12vbYQVrjLwiHTMGsAGAcjze8KeVQUqemqGiVTWF63Xq6F7ABUp, address:145.79.227.20:4448): protocol: write tcp 188.166.225.170:45110->145.79.227.20:4448: use of closed network connection; write tcp 188.166.225.170:45110->145.79.227.20:4448: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45110->145.79.227.20:4448: use of closed network connection Sep 6 09:37:01.862137534 (187h28m58.542333104s) [noise=true , address=145.79.227.48:4303 , piece_id=MLUSJEDNB632B4NIYKCN252UO2A5L5Q2PVIVR5OAIA4GE5DT6PDQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:12gJKrCBLJWN1SBKibfeqBtcqqiv4veKTGmtnycjZyb3VN9kwoT, address:145.79.227.48:4303): protocol: read tcp 188.166.225.170:45302->145.79.227.48:4303: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:01.862275349 (187h28m58.542470918s) [noise=true , address=145.79.227.37:4592 , piece_id=MN2FMLCC6IBON2PRUVZJRKJDP6CE67J4PXACPUQMCAZWBETQEOLQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:1MCAtZyJzJkfBzV8rDoZCgQPJ7PCBFiQNgCNyebQYkrEusRR4A, address:145.79.227.37:4592): protocol: read tcp 188.166.225.170:39882->145.79.227.37:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:01.862357436 (187h28m58.542553005s) [noise=true , address=145.79.227.44:4209 , piece_id=3D33RY3YVSKFXDQISLWKJJZ6SZAZMCF74UWMRIWGN2WJ5BVV7IDA, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:12EmE2BgEZTG1LTsZ5p9ghqDSfg5ZQATt31Tqg5jZ5KQE2UjDnR, address:145.79.227.44:4209): protocol: write tcp 188.166.225.170:57554->145.79.227.44:4209: use of closed network connection; write tcp 188.166.225.170:57554->145.79.227.44:4209: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57554->145.79.227.44:4209: use of closed network connection Sep 6 09:37:01.862451145 (187h28m58.542646714s) [noise=true , address=145.79.227.33:4886 , piece_id=4UFZC7W2XBULOL6WHKSISPLKB5I2T6LZUZEVETUIV6IKKVZTP5SA, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR, address:145.79.227.33:4886): protocol: read tcp 188.166.225.170:36698->145.79.227.33:4886: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:01.862461985 (187h28m58.542657564s) [noise=true , address=77.90.31.15:4726 , piece_id=YS6LW63RZ7A6SOWLU5EWJO73JNMIA3FCUM5QY3QUUN3XAGYB2Z2Q, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:12kMaukCkCJSc3ukAZu3thLgzXGGzNq44dGyEjCPqoY3PSYVEm3, address:77.90.31.15:4726): protocol: read tcp 188.166.225.170:56242->77.90.31.15:4726: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:01.862521620 (187h28m58.542717199s) [noise=true , address=145.79.227.3:4184 , piece_id=V2RFY7356GIAIUQVBWVUUFFB5MXSF4DJ6VF3JCLQIOYWX545OIUQ, seg_pos={0 0}, upload=164971]: Done uploading piece. err: ecclient: upload failed (node:17JJjwwgSn7A6BS7n1a7fphX8oTEQpXmad25ifM2nDhWhPv4Ls, address:145.79.227.3:4184): protocol: read tcp 188.166.225.170:37942->145.79.227.3:4184: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:37:01.862563921 (187h28m58.5427595s) [seg_pos={0 0}, upload=164971]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil> Duration: 1m1.434725631s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 30.000227047s FAIL (2025-09-06 09:37:00 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 30.000255891s edge gateway us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 872.49844ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-06 09:36:58 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 872.524108ms edge linksharing us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 6 out of 60 is failed (threshold=3) [................................................XXX....X.XX.] duration: 6.7736764s remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login FAIL (2025-09-06 09:36:55 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 6.773694244s core satellite us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................................X..X....X.X......] duration: 6.143917502s FAIL (2025-09-06 09:36:53 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 47) Duration: 6.143943543s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [..........................................XXXXXX.....XXX.X.X] duration: 30.001036707s FAIL (2025-09-06 09:36:50 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:36:25.642132301 (187h28m22.322327871s) [upload=164973]: Starting upload Sep 6 09:36:25.642237984 (187h28m22.322433553s) [upload=164973]: Done starting upload Sep 6 09:36:25.642290845 (187h28m22.322486414s) [upload=164973]: Uploading segments... Sep 6 09:36:25.642329599 (187h28m22.322525168s) [upload=164973]: Got next segment. Inline: true Sep 6 09:36:25.642333516 (187h28m22.322529085s) [upload=164973]: Waiting for error group managing segments... Sep 6 09:36:25.642368694 (187h28m22.322564263s) [upload=164973]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:36:25.642373233 (187h28m22.322568802s) [upload=164973]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:36:25.642376078 (187h28m22.322571647s) [upload=164973]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:36:36.637846177 (187h28m33.318041746s) [upload=164973]: Done uploading segments... Sep 6 09:36:36.637867978 (187h28m33.318063547s) [upload=164973]: Upload finished. err: <nil> Duration:* 30.001057436s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 6 out of 60 is failed (threshold=3) [.................................................XXX....X.XX] duration: 1m1.87507713s remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login FAIL (2025-09-06 09:36:48 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 1m1.875088862s core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.000987222s FAIL (2025-09-06 09:36:37 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001010127s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XX...XX....] duration: 1m0.000598693s FAIL (2025-09-06 09:36:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:31:41.360710528 (187h23m38.040906097s) [upload=164914]: Starting upload Sep 6 09:31:41.360783447 (187h23m38.040979016s) [upload=164914]: Done starting upload Sep 6 09:31:41.360862178 (187h23m38.041057747s) [upload=164914]: Uploading segments... Sep 6 09:31:41.360909148 (187h23m38.041104747s) [upload=164914]: Got next segment. Inline: true Sep 6 09:31:41.360914728 (187h23m38.041110297s) [upload=164914]: Waiting for error group managing segments... Sep 6 09:31:41.360984312 (187h23m38.041179881s) [upload=164914]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:31:41.360989862 (187h23m38.041185431s) [upload=164914]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:31:41.360992587 (187h23m38.041188156s) [upload=164914]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:32:39.493273014 (187h24m36.173468573s) [upload=164914]: Done uploading segments... Sep 6 09:32:39.493295748 (187h24m36.173491317s) [upload=164914]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1m0.000622429s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [...........................................X.X.X.X.XX.XXX.X.] duration: 12.622747988s FAIL (2025-09-06 09:36:37 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 12.622765781s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 14 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXX...XX] duration: 11.451158907s FAIL (2025-09-06 09:36:37 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 11.45118689s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [......................................................X.X..X] duration: 30.001568734s target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-06 09:36:35 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 30.001602498s edge gateway vhost us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 836.08475ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-06 09:36:27 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 836.107164ms edge linksharing us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [............................................X.X.X.X.XX.XXX.X] duration: 30.000498396s FAIL (2025-09-06 09:36:24 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000511991s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)