Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.987141219s trace: https://jaeger.dev.storj.io/trace/55634d9c1d55ff67 FAIL (2025-12-09 07:52:40 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.987176326s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 3 out of 60 is failed (threshold=3) [........................XX..X...............................] duration: 687.752765ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/43834d9fac36e837 FAIL (2025-12-09 07:52:39 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 687.780348ms core satellite us1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 467.826492ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt trace: https://jaeger.dev.storj.io/trace/51a280a8635ce0b9 FAIL (2025-12-09 07:52:34 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 467.853392ms edge linksharing us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X....................X.......] duration: 5.844996758s trace: https://jaeger.dev.storj.io/trace/7fe88be41ea8781e FAIL (2025-12-09 07:52:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng piece... Dec 9 07:45:33.502533729 (88h16m33.373616774s) [noise=true , address=77.90.31.11:4580 , piece_id=UDQZXQNUWDEILKNR42DLKTYVIJX3M7Q2YH3KOCBETUEBQZZ2WU4A, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.503057677 (88h16m33.374140712s) [noise=true , address=77.90.31.17:4797 , piece_id=XQINPPZJLK6HMR33M2SGC3EHJKL4XX7KZRWNM532PZHD3DEQ4M3Q, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.503561135 (88h16m33.37464417s) [noise=true , address=45.91.116.6:4987 , piece_id=JEQK2JKSRMQLGLNLVPBCPS6MCRYOJ7W7IEB4HBTBYFLIGO3CRXKQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.504037222 (88h16m33.375120257s) [noise=true , address=77.90.31.10:4546 , piece_id=SNWHQLYV5CBLDDT2BHM7IKY25CHDCJFTTT2APV2IN7RQQIVG7TZA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.504531994 (88h16m33.375615039s) [noise=true , address=77.90.31.21:4895 , piece_id=SW37FGLFYCZCFUVZL7BISFI3FQS2G2RXXYLD2IPOH75UE475HCTQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.505014954 (88h16m33.376098029s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.505576262 (88h16m33.376659307s) [noise=true , address=45.91.116.8:4071 , piece_id=INHPP6K7NOVPOV2G6CAUFSIKF2Q3KB5SOEE53MVU4MUDPMLSLC4A, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.506942875 (88h16m33.37802591s) [noise=true , address=45.91.116.11:4133 , piece_id=V72CQI43EBT2QWQKAC5GU2MSB7UKF7G5NIGMTLS7EOLOBGEG4YQQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.508063960 (88h16m33.379147005s) [noise=true , address=77.90.31.16:4749 , piece_id=B7JNUSR74QRRNXEFUYILTIMALV5DN75TX7P67VQ6TDGXLBLFDCGQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.508207313 (88h16m33.379290358s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.509568444 (88h16m33.38065149s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.511408599 (88h16m33.382491644s) [noise=true , address=77.90.31.19:4875 , piece_id=EA3XPU4W6DRDZMYWY2XW4IQ35DMCQEV7XQSNHJN33H24K7MJVOKA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.513502646 (88h16m33.384585681s) [noise=true , address=77.90.31.14:4683 , piece_id=FAS45YECJULRHU63VG4NFL3NPFCZPQW7IBELFWGVGMIE7YQS3J6Q, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.515541418 (88h16m33.386624484s) [noise=true , address=145.79.227.23:4565 , piece_id=PRBWTR7SVNCCCP5ERNUPB6TJOPXSPYFSZJIOL3RLPC4KRVORYWXQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.517269299 (88h16m33.388352334s) [noise=true , address=145.79.227.52:4452 , piece_id=X2RBQGFCIHR4JAJTJJ3FX4CA6PQIQUA2YHYJUANV7JIU5PKAVNLA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.518893461 (88h16m33.389976496s) [noise=true , address=77.90.31.15:4732 , piece_id=HYM7NBL4TFURODTBMWBJ7J23DON4WNYFJQ4Q3WNEBN7KCZYV3ORA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:34.970483840 (88h16m34.841566885s) [upload=77044]: Next returned nil segment Dec 9 07:45:34.970497998 (88h16m34.841581043s) [upload=77044]: Waiting for error group managing segments... Dec 9 07:45:35.375513876 (88h16m35.246596921s) [noise=true , address=145.79.227.23:4565 , piece_id=PRBWTR7SVNCCCP5ERNUPB6TJOPXSPYFSZJIOL3RLPC4KRVORYWXQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.378851702 (88h16m35.249934747s) [noise=true , address=145.79.227.52:4452 , piece_id=X2RBQGFCIHR4JAJTJJ3FX4CA6PQIQUA2YHYJUANV7JIU5PKAVNLA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.452093509 (88h16m35.323176544s) [noise=true , address=77.90.31.21:4895 , piece_id=SW37FGLFYCZCFUVZL7BISFI3FQS2G2RXXYLD2IPOH75UE475HCTQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.484823964 (88h16m35.355906999s) [noise=true , address=77.90.31.19:4875 , piece_id=EA3XPU4W6DRDZMYWY2XW4IQ35DMCQEV7XQSNHJN33H24K7MJVOKA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.501356315 (88h16m35.37243935s) [noise=true , address=77.90.31.16:4749 , piece_id=B7JNUSR74QRRNXEFUYILTIMALV5DN75TX7P67VQ6TDGXLBLFDCGQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.506575192 (88h16m35.377658238s) [noise=true , address=77.90.31.14:4683 , piece_id=FAS45YECJULRHU63VG4NFL3NPFCZPQW7IBELFWGVGMIE7YQS3J6Q, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.511021099 (88h16m35.382104134s) [noise=true , address=77.90.31.17:4797 , piece_id=XQINPPZJLK6HMR33M2SGC3EHJKL4XX7KZRWNM532PZHD3DEQ4M3Q, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.519588893 (88h16m35.390671938s) [noise=true , address=77.90.31.11:4580 , piece_id=UDQZXQNUWDEILKNR42DLKTYVIJX3M7Q2YH3KOCBETUEBQZZ2WU4A, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.540564915 (88h16m35.41164795s) [noise=true , address=45.91.116.11:4133 , piece_id=V72CQI43EBT2QWQKAC5GU2MSB7UKF7G5NIGMTLS7EOLOBGEG4YQQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.555394233 (88h16m35.426477278s) [noise=true , address=77.90.31.10:4546 , piece_id=SNWHQLYV5CBLDDT2BHM7IKY25CHDCJFTTT2APV2IN7RQQIVG7TZA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.565337939 (88h16m35.436420984s) [noise=true , address=45.91.116.8:4071 , piece_id=INHPP6K7NOVPOV2G6CAUFSIKF2Q3KB5SOEE53MVU4MUDPMLSLC4A, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.576688943 (88h16m35.447771978s) [noise=true , address=77.90.31.15:4732 , piece_id=HYM7NBL4TFURODTBMWBJ7J23DON4WNYFJQ4Q3WNEBN7KCZYV3ORA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.589009313 (88h16m35.460092368s) [noise=true , address=45.91.116.13:4181 , piece_id=X6IEEUKM5GDXZD2URW6YAPSN4DXWOCD2NK56K5PFFAYEHL2RTJUQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.619301786 (88h16m35.490384821s) [noise=true , address=45.91.116.6:4987 , piece_id=JEQK2JKSRMQLGLNLVPBCPS6MCRYOJ7W7IEB4HBTBYFLIGO3CRXKQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.619323929 (88h16m35.490406964s) [seg_pos={0 0}, upload=77044]: Segment reached optimal threshold of 14 pieces. Dec 9 07:45:35.619554758 (88h16m35.490637793s) [noise=true , address=45.91.116.12:4231 , piece_id=HZ7JTNUJBWREEUHR3A6D5GG5ENKCA4MXAHCCNMI4MU6ZHKZUC6MQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NmgZdLWKjrDMmNcgmnzH475akX4WtA9zGTLxZgVW7JpaecBjq): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619564476 (88h16m35.490647531s) [noise=true , address=45.91.116.12:4231 , piece_id=HZ7JTNUJBWREEUHR3A6D5GG5ENKCA4MXAHCCNMI4MU6ZHKZUC6MQ, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619577912 (88h16m35.490660947s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619583001 (88h16m35.490666037s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619751342 (88h16m35.490834377s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127KQ2X6VXddEeFRMau7MVGwtTsUmFsm8hTYDR2A6B4A1oVKfyo): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619757253 (88h16m35.490840298s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619853827 (88h16m35.490936862s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection Dec 9 07:45:35.619862484 (88h16m35.490945519s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619883484 (88h16m35.490966519s) [seg_pos={0 0}, upload=77044]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 07:45:35.619956242 (88h16m35.491039288s) [upload=77044]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 07:45:35.619961382 (88h16m35.491044417s) [upload=77044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 07:47:56.102293256 (88h18m55.973376291s) [upload=77044]: Done uploading segments... Dec 9 07:47:56.102346187 (88h18m55.973429222s) [upload=77044]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.845023649s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 5.697111145s trace: https://jaeger.dev.storj.io/trace/5f10a1154ce06783 FAIL (2025-12-09 07:52:32 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.69713967s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [..........................X.XX.XX..X..................X.....] duration: 3.59321475s trace: https://jaeger.dev.storj.io/trace/610b5afc07f67951 FAIL (2025-12-09 07:52:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.593240788s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X...................X......................................] duration: 5.778144169s trace: https://jaeger.dev.storj.io/trace/59c049b2e292a646 FAIL (2025-12-09 07:52:25 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.778173493s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................XXX.X...................................] duration: 5.640438941s trace: https://jaeger.dev.storj.io/trace/34e772a771ff36b2 FAIL (2025-12-09 07:52:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== :29:22.029442363 (88h0m21.900525398s) [noise=true , address=77.90.31.10:4519 , piece_id=PLAP7Q4BYYOUCI53OQKNR74TS4XCPEI6GTH5CETYOWWCEPIDCTEA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.029471198 (88h0m21.900554233s) [noise=true , address=145.79.227.20:4438 , piece_id=NC5ZFPAH575LULPHREYIQXSW44HHPJEUAVCM2RZYOACUWYOD6D6A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.030555563 (88h0m21.901638608s) [noise=true , address=145.79.227.26:4654 , piece_id=LH5BDOW3HG7P7VSLJ3EUBPLOPK3DPAXSJB5RMA5KFEMDQ3R52W3A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.031156617 (88h0m21.902239662s) [noise=true , address=77.90.31.10:4520 , piece_id=5JGFSZCUORYTBSH5IEYGLGW3GJ7IIKZZBUNG7ZYI2DWKCAC7FN4A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.031594381 (88h0m21.902677416s) [noise=true , address=145.79.227.39:4014 , piece_id=QSO5DWCCBHO7WP2XJO2Y7RP3WMSJPIEZZWNKYNB6CHJNXXKTXUQQ, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032034670 (88h0m21.903117715s) [noise=true , address=145.79.227.2:4114 , piece_id=KNMNK52ESYIBOSEF4HTDJOFJSOXHXQWRUI5URRQ6WRQOFGWM47SA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032167963 (88h0m21.903250998s) [noise=true , address=145.79.227.40:4049 , piece_id=OCQVF3EKPS273EYHIPTFDMIBPQF63OITQVS6RDUNLCOCRNWBS3EQ, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032800087 (88h0m21.903883132s) [noise=true , address=145.79.227.32:4834 , piece_id=LZGJJKA7LBTTBN26ZKVJKIMIQJDOXZVEIEUIL4Y73RPOFGJZOAOA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:23.836496806 (88h0m23.707579841s) [upload=76810]: Next returned nil segment Dec 9 07:29:23.836512486 (88h0m23.707595531s) [upload=76810]: Waiting for error group managing segments... Dec 9 07:29:24.124325638 (88h0m23.995408683s) [noise=true , address=77.90.31.10:4519 , piece_id=PLAP7Q4BYYOUCI53OQKNR74TS4XCPEI6GTH5CETYOWWCEPIDCTEA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.114522896 (88h0m24.985605941s) [noise=true , address=145.79.227.21:4475 , piece_id=ED4VS3YEIFPJGGKW36BJKVLT3EXGVOFG3SD23LEPPFIAOOKHUD4Q, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.377083973 (88h0m25.248167008s) [noise=true , address=145.79.227.41:4086 , piece_id=EZUBNMEUWTNJBXJBWJYXFJQEPFFVQOUHZAWVJW6NCWKGRKAS2PSA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.480466423 (88h0m25.351549468s) [noise=true , address=145.79.227.42:4121 , piece_id=7I2TXNXZM2KM5TEDDF7Q5DDUSBAPBE3WWFTTJFGNM5JVVCTQNU5Q, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:26.767037853 (88h0m26.638120908s) [noise=true , address=77.90.31.10:4520 , piece_id=5JGFSZCUORYTBSH5IEYGLGW3GJ7IIKZZBUNG7ZYI2DWKCAC7FN4A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:49.689337786 (88h0m49.560420841s) [noise=true , address=145.79.227.40:4049 , piece_id=OCQVF3EKPS273EYHIPTFDMIBPQF63OITQVS6RDUNLCOCRNWBS3EQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection; write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection Dec 9 07:29:49.689500304 (88h0m49.56058335s) [noise=true , address=145.79.227.34:4907 , piece_id=UG6M3T655YGW4OGBYEVOLQVR7U5XWXPDDNXBZAPTHYP2DYWDRRYA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS, address:145.79.227.34:4907): protocol: write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection; write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection Dec 9 07:29:49.689541303 (88h0m49.560624348s) [noise=true , address=145.79.227.33:4870 , piece_id=6Q5XSDCJCDRWHPO7JWCDNPQM5JN2ESDCRXWY5V23AGA6Q4C5Y7HQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt, address:145.79.227.33:4870): protocol: write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection; write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection Dec 9 07:29:49.689654439 (88h0m49.560737474s) [noise=true , address=145.79.227.29:4943 , piece_id=FQ2FNNQR6QBWTJOIQBIOU7BSWIA6ZZMDWC4DE42NUNR5Z7ZDGCGQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection; write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection Dec 9 07:29:49.689700015 (88h0m49.56078306s) [noise=true , address=145.79.227.42:4122 , piece_id=5WKTAWBKWCZVU3YX4RSF4FQHQABNCZUHTXMTK3AIHUAGYDQXQYVQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:124d52DhtbtwipVpZuWdpFTQAttR18hy1K1BbokNRX4R1bLtzTi, address:145.79.227.42:4122): protocol: write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection; write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection Dec 9 07:29:49.689861212 (88h0m49.560944247s) [noise=true , address=145.79.227.39:4014 , piece_id=QSO5DWCCBHO7WP2XJO2Y7RP3WMSJPIEZZWNKYNB6CHJNXXKTXUQQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection; write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection Dec 9 07:29:49.689914644 (88h0m49.560997689s) [noise=true , address=145.79.227.28:4725 , piece_id=AGDXVUHDFM7DC2HNW3YUKREAROODLZBIVUMTFYBY3CXYQ7HSYGWA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection Dec 9 07:29:49.690003323 (88h0m49.561086368s) [noise=true , address=145.79.227.32:4834 , piece_id=LZGJJKA7LBTTBN26ZKVJKIMIQJDOXZVEIEUIL4Y73RPOFGJZOAOA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection; write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection Dec 9 07:29:49.690090949 (88h0m49.561173994s) [noise=true , address=145.79.227.26:4654 , piece_id=LH5BDOW3HG7P7VSLJ3EUBPLOPK3DPAXSJB5RMA5KFEMDQ3R52W3A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection Dec 9 07:29:49.690262546 (88h0m49.561345591s) [noise=true , address=145.79.227.2:4114 , piece_id=KNMNK52ESYIBOSEF4HTDJOFJSOXHXQWRUI5URRQ6WRQOFGWM47SA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection Dec 9 07:29:49.690582726 (88h0m49.561665772s) [noise=true , address=145.79.227.20:4438 , piece_id=NC5ZFPAH575LULPHREYIQXSW44HHPJEUAVCM2RZYOACUWYOD6D6A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT, address:145.79.227.20:4438): protocol: read tcp 188.166.225.170:50466->145.79.227.20:4438: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 07:29:49.690631769 (88h0m49.561714805s) [seg_pos={0 0}, upload=76810]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Dec 9 07:29:50.737500571 (88h0m50.608583616s) [upload=76810]: Done uploading segments... Dec 9 07:29:50.737550175 (88h0m50.60863321s) [upload=76810]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.640468756s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................XXX..................................] duration: 2.323577062s trace: https://jaeger.dev.storj.io/trace/178b4cba623ae119 FAIL (2025-12-09 07:52:22 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.323735053s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.................X...........XX..X.......X...X..............] duration: 5.834235066s trace: https://jaeger.dev.storj.io/trace/76e27d463b59cd34 FAIL (2025-12-09 07:52:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.834269851s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.123338144s trace: https://jaeger.dev.storj.io/trace/50f433cc1e60731c FAIL (2025-12-09 07:52:12 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.123370296s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.951757767s trace: https://jaeger.dev.storj.io/trace/55634d9c1d55ff67 FAIL (2025-12-09 07:52:05 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.95179069s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.902088353s trace: https://jaeger.dev.storj.io/trace/42dcab7a5e240fd8 FAIL (2025-12-09 07:51:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 9 07:28:09.412200246 (87h59m9.283283281s) [upload=76792]: Starting upload Dec 9 07:28:09.412257335 (87h59m9.283340371s) [upload=76792]: Done starting upload Dec 9 07:28:09.412337338 (87h59m9.283420383s) [upload=76792]: Uploading segments... Dec 9 07:28:09.412384597 (87h59m9.283467632s) [upload=76792]: Got next segment. Inline: true Dec 9 07:28:09.412398334 (87h59m9.283481389s) [upload=76792]: Waiting for error group managing segments... Dec 9 07:28:09.412446225 (87h59m9.28352927s) [upload=76792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 07:28:09.412452456 (87h59m9.283535491s) [upload=76792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Dec 9 07:28:09.412455432 (87h59m9.283538477s) [upload=76792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 07:28:19.867794892 (87h59m19.738877927s) [upload=76792]: Done uploading segments... Dec 9 07:28:19.867843895 (87h59m19.73892694s) [upload=76792]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.902118412s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..................X...........XX..X.......X...X.............] duration: 5.667490191s trace: https://jaeger.dev.storj.io/trace/76e27d463b59cd34 FAIL (2025-12-09 07:51:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.667648893s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................XXX.X..................................] duration: 5.619225396s trace: https://jaeger.dev.storj.io/trace/34e772a771ff36b2 FAIL (2025-12-09 07:51:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== :29:22.029442363 (88h0m21.900525398s) [noise=true , address=77.90.31.10:4519 , piece_id=PLAP7Q4BYYOUCI53OQKNR74TS4XCPEI6GTH5CETYOWWCEPIDCTEA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.029471198 (88h0m21.900554233s) [noise=true , address=145.79.227.20:4438 , piece_id=NC5ZFPAH575LULPHREYIQXSW44HHPJEUAVCM2RZYOACUWYOD6D6A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.030555563 (88h0m21.901638608s) [noise=true , address=145.79.227.26:4654 , piece_id=LH5BDOW3HG7P7VSLJ3EUBPLOPK3DPAXSJB5RMA5KFEMDQ3R52W3A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.031156617 (88h0m21.902239662s) [noise=true , address=77.90.31.10:4520 , piece_id=5JGFSZCUORYTBSH5IEYGLGW3GJ7IIKZZBUNG7ZYI2DWKCAC7FN4A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.031594381 (88h0m21.902677416s) [noise=true , address=145.79.227.39:4014 , piece_id=QSO5DWCCBHO7WP2XJO2Y7RP3WMSJPIEZZWNKYNB6CHJNXXKTXUQQ, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032034670 (88h0m21.903117715s) [noise=true , address=145.79.227.2:4114 , piece_id=KNMNK52ESYIBOSEF4HTDJOFJSOXHXQWRUI5URRQ6WRQOFGWM47SA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032167963 (88h0m21.903250998s) [noise=true , address=145.79.227.40:4049 , piece_id=OCQVF3EKPS273EYHIPTFDMIBPQF63OITQVS6RDUNLCOCRNWBS3EQ, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032800087 (88h0m21.903883132s) [noise=true , address=145.79.227.32:4834 , piece_id=LZGJJKA7LBTTBN26ZKVJKIMIQJDOXZVEIEUIL4Y73RPOFGJZOAOA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:23.836496806 (88h0m23.707579841s) [upload=76810]: Next returned nil segment Dec 9 07:29:23.836512486 (88h0m23.707595531s) [upload=76810]: Waiting for error group managing segments... Dec 9 07:29:24.124325638 (88h0m23.995408683s) [noise=true , address=77.90.31.10:4519 , piece_id=PLAP7Q4BYYOUCI53OQKNR74TS4XCPEI6GTH5CETYOWWCEPIDCTEA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.114522896 (88h0m24.985605941s) [noise=true , address=145.79.227.21:4475 , piece_id=ED4VS3YEIFPJGGKW36BJKVLT3EXGVOFG3SD23LEPPFIAOOKHUD4Q, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.377083973 (88h0m25.248167008s) [noise=true , address=145.79.227.41:4086 , piece_id=EZUBNMEUWTNJBXJBWJYXFJQEPFFVQOUHZAWVJW6NCWKGRKAS2PSA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.480466423 (88h0m25.351549468s) [noise=true , address=145.79.227.42:4121 , piece_id=7I2TXNXZM2KM5TEDDF7Q5DDUSBAPBE3WWFTTJFGNM5JVVCTQNU5Q, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:26.767037853 (88h0m26.638120908s) [noise=true , address=77.90.31.10:4520 , piece_id=5JGFSZCUORYTBSH5IEYGLGW3GJ7IIKZZBUNG7ZYI2DWKCAC7FN4A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:49.689337786 (88h0m49.560420841s) [noise=true , address=145.79.227.40:4049 , piece_id=OCQVF3EKPS273EYHIPTFDMIBPQF63OITQVS6RDUNLCOCRNWBS3EQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection; write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection Dec 9 07:29:49.689500304 (88h0m49.56058335s) [noise=true , address=145.79.227.34:4907 , piece_id=UG6M3T655YGW4OGBYEVOLQVR7U5XWXPDDNXBZAPTHYP2DYWDRRYA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS, address:145.79.227.34:4907): protocol: write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection; write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection Dec 9 07:29:49.689541303 (88h0m49.560624348s) [noise=true , address=145.79.227.33:4870 , piece_id=6Q5XSDCJCDRWHPO7JWCDNPQM5JN2ESDCRXWY5V23AGA6Q4C5Y7HQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt, address:145.79.227.33:4870): protocol: write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection; write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection Dec 9 07:29:49.689654439 (88h0m49.560737474s) [noise=true , address=145.79.227.29:4943 , piece_id=FQ2FNNQR6QBWTJOIQBIOU7BSWIA6ZZMDWC4DE42NUNR5Z7ZDGCGQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection; write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection Dec 9 07:29:49.689700015 (88h0m49.56078306s) [noise=true , address=145.79.227.42:4122 , piece_id=5WKTAWBKWCZVU3YX4RSF4FQHQABNCZUHTXMTK3AIHUAGYDQXQYVQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:124d52DhtbtwipVpZuWdpFTQAttR18hy1K1BbokNRX4R1bLtzTi, address:145.79.227.42:4122): protocol: write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection; write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection Dec 9 07:29:49.689861212 (88h0m49.560944247s) [noise=true , address=145.79.227.39:4014 , piece_id=QSO5DWCCBHO7WP2XJO2Y7RP3WMSJPIEZZWNKYNB6CHJNXXKTXUQQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection; write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection Dec 9 07:29:49.689914644 (88h0m49.560997689s) [noise=true , address=145.79.227.28:4725 , piece_id=AGDXVUHDFM7DC2HNW3YUKREAROODLZBIVUMTFYBY3CXYQ7HSYGWA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection Dec 9 07:29:49.690003323 (88h0m49.561086368s) [noise=true , address=145.79.227.32:4834 , piece_id=LZGJJKA7LBTTBN26ZKVJKIMIQJDOXZVEIEUIL4Y73RPOFGJZOAOA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection; write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection Dec 9 07:29:49.690090949 (88h0m49.561173994s) [noise=true , address=145.79.227.26:4654 , piece_id=LH5BDOW3HG7P7VSLJ3EUBPLOPK3DPAXSJB5RMA5KFEMDQ3R52W3A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection Dec 9 07:29:49.690262546 (88h0m49.561345591s) [noise=true , address=145.79.227.2:4114 , piece_id=KNMNK52ESYIBOSEF4HTDJOFJSOXHXQWRUI5URRQ6WRQOFGWM47SA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection Dec 9 07:29:49.690582726 (88h0m49.561665772s) [noise=true , address=145.79.227.20:4438 , piece_id=NC5ZFPAH575LULPHREYIQXSW44HHPJEUAVCM2RZYOACUWYOD6D6A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT, address:145.79.227.20:4438): protocol: read tcp 188.166.225.170:50466->145.79.227.20:4438: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 07:29:49.690631769 (88h0m49.561714805s) [seg_pos={0 0}, upload=76810]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Dec 9 07:29:50.737500571 (88h0m50.608583616s) [upload=76810]: Done uploading segments... Dec 9 07:29:50.737550175 (88h0m50.60863321s) [upload=76810]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.619263769s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X...................X.....................................] duration: 5.861806396s trace: https://jaeger.dev.storj.io/trace/59c049b2e292a646 FAIL (2025-12-09 07:51:49 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.861836244s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [...........................X.XX.XX..X..................X....] duration: 3.553120702s trace: https://jaeger.dev.storj.io/trace/610b5afc07f67951 FAIL (2025-12-09 07:51:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.553161018s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 465.335398ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt trace: https://jaeger.dev.storj.io/trace/51a280a8635ce0b9 FAIL (2025-12-09 07:51:44 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 465.370003ms edge linksharing us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [........................XXX.................................] duration: 1.729464735s trace: https://jaeger.dev.storj.io/trace/178b4cba623ae119 FAIL (2025-12-09 07:51:43 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 1.729489463s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 3 out of 60 is failed (threshold=3) [.........................XX..X..............................] duration: 683.18189ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/43834d9fac36e837 FAIL (2025-12-09 07:51:29 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 683.210344ms core satellite us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 6.003761039s trace: https://jaeger.dev.storj.io/trace/5f10a1154ce06783 FAIL (2025-12-09 07:51:27 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.003781667s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.920009703s trace: https://jaeger.dev.storj.io/trace/55634d9c1d55ff67 FAIL (2025-12-09 07:51:25 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.920043388s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.042565315s trace: https://jaeger.dev.storj.io/trace/50f433cc1e60731c FAIL (2025-12-09 07:51:25 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.042595963s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X....................X......] duration: 6.540858504s trace: https://jaeger.dev.storj.io/trace/7fe88be41ea8781e FAIL (2025-12-09 07:51:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng piece... Dec 9 07:45:33.502533729 (88h16m33.373616774s) [noise=true , address=77.90.31.11:4580 , piece_id=UDQZXQNUWDEILKNR42DLKTYVIJX3M7Q2YH3KOCBETUEBQZZ2WU4A, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.503057677 (88h16m33.374140712s) [noise=true , address=77.90.31.17:4797 , piece_id=XQINPPZJLK6HMR33M2SGC3EHJKL4XX7KZRWNM532PZHD3DEQ4M3Q, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.503561135 (88h16m33.37464417s) [noise=true , address=45.91.116.6:4987 , piece_id=JEQK2JKSRMQLGLNLVPBCPS6MCRYOJ7W7IEB4HBTBYFLIGO3CRXKQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.504037222 (88h16m33.375120257s) [noise=true , address=77.90.31.10:4546 , piece_id=SNWHQLYV5CBLDDT2BHM7IKY25CHDCJFTTT2APV2IN7RQQIVG7TZA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.504531994 (88h16m33.375615039s) [noise=true , address=77.90.31.21:4895 , piece_id=SW37FGLFYCZCFUVZL7BISFI3FQS2G2RXXYLD2IPOH75UE475HCTQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.505014954 (88h16m33.376098029s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.505576262 (88h16m33.376659307s) [noise=true , address=45.91.116.8:4071 , piece_id=INHPP6K7NOVPOV2G6CAUFSIKF2Q3KB5SOEE53MVU4MUDPMLSLC4A, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.506942875 (88h16m33.37802591s) [noise=true , address=45.91.116.11:4133 , piece_id=V72CQI43EBT2QWQKAC5GU2MSB7UKF7G5NIGMTLS7EOLOBGEG4YQQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.508063960 (88h16m33.379147005s) [noise=true , address=77.90.31.16:4749 , piece_id=B7JNUSR74QRRNXEFUYILTIMALV5DN75TX7P67VQ6TDGXLBLFDCGQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.508207313 (88h16m33.379290358s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.509568444 (88h16m33.38065149s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.511408599 (88h16m33.382491644s) [noise=true , address=77.90.31.19:4875 , piece_id=EA3XPU4W6DRDZMYWY2XW4IQ35DMCQEV7XQSNHJN33H24K7MJVOKA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.513502646 (88h16m33.384585681s) [noise=true , address=77.90.31.14:4683 , piece_id=FAS45YECJULRHU63VG4NFL3NPFCZPQW7IBELFWGVGMIE7YQS3J6Q, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.515541418 (88h16m33.386624484s) [noise=true , address=145.79.227.23:4565 , piece_id=PRBWTR7SVNCCCP5ERNUPB6TJOPXSPYFSZJIOL3RLPC4KRVORYWXQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.517269299 (88h16m33.388352334s) [noise=true , address=145.79.227.52:4452 , piece_id=X2RBQGFCIHR4JAJTJJ3FX4CA6PQIQUA2YHYJUANV7JIU5PKAVNLA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.518893461 (88h16m33.389976496s) [noise=true , address=77.90.31.15:4732 , piece_id=HYM7NBL4TFURODTBMWBJ7J23DON4WNYFJQ4Q3WNEBN7KCZYV3ORA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:34.970483840 (88h16m34.841566885s) [upload=77044]: Next returned nil segment Dec 9 07:45:34.970497998 (88h16m34.841581043s) [upload=77044]: Waiting for error group managing segments... Dec 9 07:45:35.375513876 (88h16m35.246596921s) [noise=true , address=145.79.227.23:4565 , piece_id=PRBWTR7SVNCCCP5ERNUPB6TJOPXSPYFSZJIOL3RLPC4KRVORYWXQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.378851702 (88h16m35.249934747s) [noise=true , address=145.79.227.52:4452 , piece_id=X2RBQGFCIHR4JAJTJJ3FX4CA6PQIQUA2YHYJUANV7JIU5PKAVNLA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.452093509 (88h16m35.323176544s) [noise=true , address=77.90.31.21:4895 , piece_id=SW37FGLFYCZCFUVZL7BISFI3FQS2G2RXXYLD2IPOH75UE475HCTQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.484823964 (88h16m35.355906999s) [noise=true , address=77.90.31.19:4875 , piece_id=EA3XPU4W6DRDZMYWY2XW4IQ35DMCQEV7XQSNHJN33H24K7MJVOKA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.501356315 (88h16m35.37243935s) [noise=true , address=77.90.31.16:4749 , piece_id=B7JNUSR74QRRNXEFUYILTIMALV5DN75TX7P67VQ6TDGXLBLFDCGQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.506575192 (88h16m35.377658238s) [noise=true , address=77.90.31.14:4683 , piece_id=FAS45YECJULRHU63VG4NFL3NPFCZPQW7IBELFWGVGMIE7YQS3J6Q, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.511021099 (88h16m35.382104134s) [noise=true , address=77.90.31.17:4797 , piece_id=XQINPPZJLK6HMR33M2SGC3EHJKL4XX7KZRWNM532PZHD3DEQ4M3Q, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.519588893 (88h16m35.390671938s) [noise=true , address=77.90.31.11:4580 , piece_id=UDQZXQNUWDEILKNR42DLKTYVIJX3M7Q2YH3KOCBETUEBQZZ2WU4A, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.540564915 (88h16m35.41164795s) [noise=true , address=45.91.116.11:4133 , piece_id=V72CQI43EBT2QWQKAC5GU2MSB7UKF7G5NIGMTLS7EOLOBGEG4YQQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.555394233 (88h16m35.426477278s) [noise=true , address=77.90.31.10:4546 , piece_id=SNWHQLYV5CBLDDT2BHM7IKY25CHDCJFTTT2APV2IN7RQQIVG7TZA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.565337939 (88h16m35.436420984s) [noise=true , address=45.91.116.8:4071 , piece_id=INHPP6K7NOVPOV2G6CAUFSIKF2Q3KB5SOEE53MVU4MUDPMLSLC4A, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.576688943 (88h16m35.447771978s) [noise=true , address=77.90.31.15:4732 , piece_id=HYM7NBL4TFURODTBMWBJ7J23DON4WNYFJQ4Q3WNEBN7KCZYV3ORA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.589009313 (88h16m35.460092368s) [noise=true , address=45.91.116.13:4181 , piece_id=X6IEEUKM5GDXZD2URW6YAPSN4DXWOCD2NK56K5PFFAYEHL2RTJUQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.619301786 (88h16m35.490384821s) [noise=true , address=45.91.116.6:4987 , piece_id=JEQK2JKSRMQLGLNLVPBCPS6MCRYOJ7W7IEB4HBTBYFLIGO3CRXKQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.619323929 (88h16m35.490406964s) [seg_pos={0 0}, upload=77044]: Segment reached optimal threshold of 14 pieces. Dec 9 07:45:35.619554758 (88h16m35.490637793s) [noise=true , address=45.91.116.12:4231 , piece_id=HZ7JTNUJBWREEUHR3A6D5GG5ENKCA4MXAHCCNMI4MU6ZHKZUC6MQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NmgZdLWKjrDMmNcgmnzH475akX4WtA9zGTLxZgVW7JpaecBjq): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619564476 (88h16m35.490647531s) [noise=true , address=45.91.116.12:4231 , piece_id=HZ7JTNUJBWREEUHR3A6D5GG5ENKCA4MXAHCCNMI4MU6ZHKZUC6MQ, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619577912 (88h16m35.490660947s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619583001 (88h16m35.490666037s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619751342 (88h16m35.490834377s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127KQ2X6VXddEeFRMau7MVGwtTsUmFsm8hTYDR2A6B4A1oVKfyo): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619757253 (88h16m35.490840298s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619853827 (88h16m35.490936862s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection Dec 9 07:45:35.619862484 (88h16m35.490945519s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619883484 (88h16m35.490966519s) [seg_pos={0 0}, upload=77044]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 07:45:35.619956242 (88h16m35.491039288s) [upload=77044]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 07:45:35.619961382 (88h16m35.491044417s) [upload=77044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 07:47:56.102293256 (88h18m55.973376291s) [upload=77044]: Done uploading segments... Dec 9 07:47:56.102346187 (88h18m55.973429222s) [upload=77044]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.540875736s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................XXX................................] duration: 1.876008295s trace: https://jaeger.dev.storj.io/trace/178b4cba623ae119 FAIL (2025-12-09 07:51:23 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 1.876037951s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................XXX.X.................................] duration: 5.592033075s trace: https://jaeger.dev.storj.io/trace/34e772a771ff36b2 FAIL (2025-12-09 07:51:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== :29:22.029442363 (88h0m21.900525398s) [noise=true , address=77.90.31.10:4519 , piece_id=PLAP7Q4BYYOUCI53OQKNR74TS4XCPEI6GTH5CETYOWWCEPIDCTEA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.029471198 (88h0m21.900554233s) [noise=true , address=145.79.227.20:4438 , piece_id=NC5ZFPAH575LULPHREYIQXSW44HHPJEUAVCM2RZYOACUWYOD6D6A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.030555563 (88h0m21.901638608s) [noise=true , address=145.79.227.26:4654 , piece_id=LH5BDOW3HG7P7VSLJ3EUBPLOPK3DPAXSJB5RMA5KFEMDQ3R52W3A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.031156617 (88h0m21.902239662s) [noise=true , address=77.90.31.10:4520 , piece_id=5JGFSZCUORYTBSH5IEYGLGW3GJ7IIKZZBUNG7ZYI2DWKCAC7FN4A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.031594381 (88h0m21.902677416s) [noise=true , address=145.79.227.39:4014 , piece_id=QSO5DWCCBHO7WP2XJO2Y7RP3WMSJPIEZZWNKYNB6CHJNXXKTXUQQ, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032034670 (88h0m21.903117715s) [noise=true , address=145.79.227.2:4114 , piece_id=KNMNK52ESYIBOSEF4HTDJOFJSOXHXQWRUI5URRQ6WRQOFGWM47SA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032167963 (88h0m21.903250998s) [noise=true , address=145.79.227.40:4049 , piece_id=OCQVF3EKPS273EYHIPTFDMIBPQF63OITQVS6RDUNLCOCRNWBS3EQ, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032800087 (88h0m21.903883132s) [noise=true , address=145.79.227.32:4834 , piece_id=LZGJJKA7LBTTBN26ZKVJKIMIQJDOXZVEIEUIL4Y73RPOFGJZOAOA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:23.836496806 (88h0m23.707579841s) [upload=76810]: Next returned nil segment Dec 9 07:29:23.836512486 (88h0m23.707595531s) [upload=76810]: Waiting for error group managing segments... Dec 9 07:29:24.124325638 (88h0m23.995408683s) [noise=true , address=77.90.31.10:4519 , piece_id=PLAP7Q4BYYOUCI53OQKNR74TS4XCPEI6GTH5CETYOWWCEPIDCTEA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.114522896 (88h0m24.985605941s) [noise=true , address=145.79.227.21:4475 , piece_id=ED4VS3YEIFPJGGKW36BJKVLT3EXGVOFG3SD23LEPPFIAOOKHUD4Q, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.377083973 (88h0m25.248167008s) [noise=true , address=145.79.227.41:4086 , piece_id=EZUBNMEUWTNJBXJBWJYXFJQEPFFVQOUHZAWVJW6NCWKGRKAS2PSA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.480466423 (88h0m25.351549468s) [noise=true , address=145.79.227.42:4121 , piece_id=7I2TXNXZM2KM5TEDDF7Q5DDUSBAPBE3WWFTTJFGNM5JVVCTQNU5Q, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:26.767037853 (88h0m26.638120908s) [noise=true , address=77.90.31.10:4520 , piece_id=5JGFSZCUORYTBSH5IEYGLGW3GJ7IIKZZBUNG7ZYI2DWKCAC7FN4A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:49.689337786 (88h0m49.560420841s) [noise=true , address=145.79.227.40:4049 , piece_id=OCQVF3EKPS273EYHIPTFDMIBPQF63OITQVS6RDUNLCOCRNWBS3EQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection; write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection Dec 9 07:29:49.689500304 (88h0m49.56058335s) [noise=true , address=145.79.227.34:4907 , piece_id=UG6M3T655YGW4OGBYEVOLQVR7U5XWXPDDNXBZAPTHYP2DYWDRRYA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS, address:145.79.227.34:4907): protocol: write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection; write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection Dec 9 07:29:49.689541303 (88h0m49.560624348s) [noise=true , address=145.79.227.33:4870 , piece_id=6Q5XSDCJCDRWHPO7JWCDNPQM5JN2ESDCRXWY5V23AGA6Q4C5Y7HQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt, address:145.79.227.33:4870): protocol: write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection; write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection Dec 9 07:29:49.689654439 (88h0m49.560737474s) [noise=true , address=145.79.227.29:4943 , piece_id=FQ2FNNQR6QBWTJOIQBIOU7BSWIA6ZZMDWC4DE42NUNR5Z7ZDGCGQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection; write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection Dec 9 07:29:49.689700015 (88h0m49.56078306s) [noise=true , address=145.79.227.42:4122 , piece_id=5WKTAWBKWCZVU3YX4RSF4FQHQABNCZUHTXMTK3AIHUAGYDQXQYVQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:124d52DhtbtwipVpZuWdpFTQAttR18hy1K1BbokNRX4R1bLtzTi, address:145.79.227.42:4122): protocol: write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection; write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection Dec 9 07:29:49.689861212 (88h0m49.560944247s) [noise=true , address=145.79.227.39:4014 , piece_id=QSO5DWCCBHO7WP2XJO2Y7RP3WMSJPIEZZWNKYNB6CHJNXXKTXUQQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection; write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection Dec 9 07:29:49.689914644 (88h0m49.560997689s) [noise=true , address=145.79.227.28:4725 , piece_id=AGDXVUHDFM7DC2HNW3YUKREAROODLZBIVUMTFYBY3CXYQ7HSYGWA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection Dec 9 07:29:49.690003323 (88h0m49.561086368s) [noise=true , address=145.79.227.32:4834 , piece_id=LZGJJKA7LBTTBN26ZKVJKIMIQJDOXZVEIEUIL4Y73RPOFGJZOAOA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection; write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection Dec 9 07:29:49.690090949 (88h0m49.561173994s) [noise=true , address=145.79.227.26:4654 , piece_id=LH5BDOW3HG7P7VSLJ3EUBPLOPK3DPAXSJB5RMA5KFEMDQ3R52W3A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection Dec 9 07:29:49.690262546 (88h0m49.561345591s) [noise=true , address=145.79.227.2:4114 , piece_id=KNMNK52ESYIBOSEF4HTDJOFJSOXHXQWRUI5URRQ6WRQOFGWM47SA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection Dec 9 07:29:49.690582726 (88h0m49.561665772s) [noise=true , address=145.79.227.20:4438 , piece_id=NC5ZFPAH575LULPHREYIQXSW44HHPJEUAVCM2RZYOACUWYOD6D6A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT, address:145.79.227.20:4438): protocol: read tcp 188.166.225.170:50466->145.79.227.20:4438: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 07:29:49.690631769 (88h0m49.561714805s) [seg_pos={0 0}, upload=76810]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Dec 9 07:29:50.737500571 (88h0m50.608583616s) [upload=76810]: Done uploading segments... Dec 9 07:29:50.737550175 (88h0m50.60863321s) [upload=76810]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.592078983s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.021058538s trace: https://jaeger.dev.storj.io/trace/42dcab7a5e240fd8 FAIL (2025-12-09 07:51:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 9 07:28:09.412200246 (87h59m9.283283281s) [upload=76792]: Starting upload Dec 9 07:28:09.412257335 (87h59m9.283340371s) [upload=76792]: Done starting upload Dec 9 07:28:09.412337338 (87h59m9.283420383s) [upload=76792]: Uploading segments... Dec 9 07:28:09.412384597 (87h59m9.283467632s) [upload=76792]: Got next segment. Inline: true Dec 9 07:28:09.412398334 (87h59m9.283481389s) [upload=76792]: Waiting for error group managing segments... Dec 9 07:28:09.412446225 (87h59m9.28352927s) [upload=76792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 07:28:09.412452456 (87h59m9.283535491s) [upload=76792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Dec 9 07:28:09.412455432 (87h59m9.283538477s) [upload=76792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 07:28:19.867794892 (87h59m19.738877927s) [upload=76792]: Done uploading segments... Dec 9 07:28:19.867843895 (87h59m19.73892694s) [upload=76792]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.021150614s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X...................X....................................] duration: 5.657169317s trace: https://jaeger.dev.storj.io/trace/59c049b2e292a646 FAIL (2025-12-09 07:51:12 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.65720215s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 3 out of 60 is failed (threshold=3) [..........................XX..X.............................] duration: 693.547909ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/43834d9fac36e837 FAIL (2025-12-09 07:51:12 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 693.577606ms core satellite us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........................XXX...............................] duration: 2.21912296s trace: https://jaeger.dev.storj.io/trace/178b4cba623ae119 FAIL (2025-12-09 07:51:07 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.219156643s edge gateway us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 473.947507ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt trace: https://jaeger.dev.storj.io/trace/51a280a8635ce0b9 FAIL (2025-12-09 07:51:01 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 473.977906ms edge linksharing us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 3 out of 60 is failed (threshold=3) [...........................XX..X............................] duration: 691.309537ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/43834d9fac36e837 FAIL (2025-12-09 07:50:56 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 691.341858ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...................X...........XX..X.......X...X............] duration: 5.686656667s trace: https://jaeger.dev.storj.io/trace/76e27d463b59cd34 FAIL (2025-12-09 07:50:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.686684691s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................XXX.X................................] duration: 5.594477539s trace: https://jaeger.dev.storj.io/trace/34e772a771ff36b2 FAIL (2025-12-09 07:50:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== :29:22.029442363 (88h0m21.900525398s) [noise=true , address=77.90.31.10:4519 , piece_id=PLAP7Q4BYYOUCI53OQKNR74TS4XCPEI6GTH5CETYOWWCEPIDCTEA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.029471198 (88h0m21.900554233s) [noise=true , address=145.79.227.20:4438 , piece_id=NC5ZFPAH575LULPHREYIQXSW44HHPJEUAVCM2RZYOACUWYOD6D6A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.030555563 (88h0m21.901638608s) [noise=true , address=145.79.227.26:4654 , piece_id=LH5BDOW3HG7P7VSLJ3EUBPLOPK3DPAXSJB5RMA5KFEMDQ3R52W3A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.031156617 (88h0m21.902239662s) [noise=true , address=77.90.31.10:4520 , piece_id=5JGFSZCUORYTBSH5IEYGLGW3GJ7IIKZZBUNG7ZYI2DWKCAC7FN4A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.031594381 (88h0m21.902677416s) [noise=true , address=145.79.227.39:4014 , piece_id=QSO5DWCCBHO7WP2XJO2Y7RP3WMSJPIEZZWNKYNB6CHJNXXKTXUQQ, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032034670 (88h0m21.903117715s) [noise=true , address=145.79.227.2:4114 , piece_id=KNMNK52ESYIBOSEF4HTDJOFJSOXHXQWRUI5URRQ6WRQOFGWM47SA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032167963 (88h0m21.903250998s) [noise=true , address=145.79.227.40:4049 , piece_id=OCQVF3EKPS273EYHIPTFDMIBPQF63OITQVS6RDUNLCOCRNWBS3EQ, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032800087 (88h0m21.903883132s) [noise=true , address=145.79.227.32:4834 , piece_id=LZGJJKA7LBTTBN26ZKVJKIMIQJDOXZVEIEUIL4Y73RPOFGJZOAOA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:23.836496806 (88h0m23.707579841s) [upload=76810]: Next returned nil segment Dec 9 07:29:23.836512486 (88h0m23.707595531s) [upload=76810]: Waiting for error group managing segments... Dec 9 07:29:24.124325638 (88h0m23.995408683s) [noise=true , address=77.90.31.10:4519 , piece_id=PLAP7Q4BYYOUCI53OQKNR74TS4XCPEI6GTH5CETYOWWCEPIDCTEA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.114522896 (88h0m24.985605941s) [noise=true , address=145.79.227.21:4475 , piece_id=ED4VS3YEIFPJGGKW36BJKVLT3EXGVOFG3SD23LEPPFIAOOKHUD4Q, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.377083973 (88h0m25.248167008s) [noise=true , address=145.79.227.41:4086 , piece_id=EZUBNMEUWTNJBXJBWJYXFJQEPFFVQOUHZAWVJW6NCWKGRKAS2PSA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.480466423 (88h0m25.351549468s) [noise=true , address=145.79.227.42:4121 , piece_id=7I2TXNXZM2KM5TEDDF7Q5DDUSBAPBE3WWFTTJFGNM5JVVCTQNU5Q, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:26.767037853 (88h0m26.638120908s) [noise=true , address=77.90.31.10:4520 , piece_id=5JGFSZCUORYTBSH5IEYGLGW3GJ7IIKZZBUNG7ZYI2DWKCAC7FN4A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:49.689337786 (88h0m49.560420841s) [noise=true , address=145.79.227.40:4049 , piece_id=OCQVF3EKPS273EYHIPTFDMIBPQF63OITQVS6RDUNLCOCRNWBS3EQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection; write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection Dec 9 07:29:49.689500304 (88h0m49.56058335s) [noise=true , address=145.79.227.34:4907 , piece_id=UG6M3T655YGW4OGBYEVOLQVR7U5XWXPDDNXBZAPTHYP2DYWDRRYA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS, address:145.79.227.34:4907): protocol: write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection; write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection Dec 9 07:29:49.689541303 (88h0m49.560624348s) [noise=true , address=145.79.227.33:4870 , piece_id=6Q5XSDCJCDRWHPO7JWCDNPQM5JN2ESDCRXWY5V23AGA6Q4C5Y7HQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt, address:145.79.227.33:4870): protocol: write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection; write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection Dec 9 07:29:49.689654439 (88h0m49.560737474s) [noise=true , address=145.79.227.29:4943 , piece_id=FQ2FNNQR6QBWTJOIQBIOU7BSWIA6ZZMDWC4DE42NUNR5Z7ZDGCGQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection; write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection Dec 9 07:29:49.689700015 (88h0m49.56078306s) [noise=true , address=145.79.227.42:4122 , piece_id=5WKTAWBKWCZVU3YX4RSF4FQHQABNCZUHTXMTK3AIHUAGYDQXQYVQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:124d52DhtbtwipVpZuWdpFTQAttR18hy1K1BbokNRX4R1bLtzTi, address:145.79.227.42:4122): protocol: write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection; write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection Dec 9 07:29:49.689861212 (88h0m49.560944247s) [noise=true , address=145.79.227.39:4014 , piece_id=QSO5DWCCBHO7WP2XJO2Y7RP3WMSJPIEZZWNKYNB6CHJNXXKTXUQQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection; write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection Dec 9 07:29:49.689914644 (88h0m49.560997689s) [noise=true , address=145.79.227.28:4725 , piece_id=AGDXVUHDFM7DC2HNW3YUKREAROODLZBIVUMTFYBY3CXYQ7HSYGWA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection Dec 9 07:29:49.690003323 (88h0m49.561086368s) [noise=true , address=145.79.227.32:4834 , piece_id=LZGJJKA7LBTTBN26ZKVJKIMIQJDOXZVEIEUIL4Y73RPOFGJZOAOA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection; write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection Dec 9 07:29:49.690090949 (88h0m49.561173994s) [noise=true , address=145.79.227.26:4654 , piece_id=LH5BDOW3HG7P7VSLJ3EUBPLOPK3DPAXSJB5RMA5KFEMDQ3R52W3A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection Dec 9 07:29:49.690262546 (88h0m49.561345591s) [noise=true , address=145.79.227.2:4114 , piece_id=KNMNK52ESYIBOSEF4HTDJOFJSOXHXQWRUI5URRQ6WRQOFGWM47SA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection Dec 9 07:29:49.690582726 (88h0m49.561665772s) [noise=true , address=145.79.227.20:4438 , piece_id=NC5ZFPAH575LULPHREYIQXSW44HHPJEUAVCM2RZYOACUWYOD6D6A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT, address:145.79.227.20:4438): protocol: read tcp 188.166.225.170:50466->145.79.227.20:4438: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 07:29:49.690631769 (88h0m49.561714805s) [seg_pos={0 0}, upload=76810]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Dec 9 07:29:50.737500571 (88h0m50.608583616s) [upload=76810]: Done uploading segments... Dec 9 07:29:50.737550175 (88h0m50.60863321s) [upload=76810]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.594505412s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........................XXX..............................] duration: 2.179881087s trace: https://jaeger.dev.storj.io/trace/178b4cba623ae119 FAIL (2025-12-09 07:50:41 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.179911053s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.013374565s trace: https://jaeger.dev.storj.io/trace/42dcab7a5e240fd8 FAIL (2025-12-09 07:50:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 9 07:28:09.412200246 (87h59m9.283283281s) [upload=76792]: Starting upload Dec 9 07:28:09.412257335 (87h59m9.283340371s) [upload=76792]: Done starting upload Dec 9 07:28:09.412337338 (87h59m9.283420383s) [upload=76792]: Uploading segments... Dec 9 07:28:09.412384597 (87h59m9.283467632s) [upload=76792]: Got next segment. Inline: true Dec 9 07:28:09.412398334 (87h59m9.283481389s) [upload=76792]: Waiting for error group managing segments... Dec 9 07:28:09.412446225 (87h59m9.28352927s) [upload=76792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 07:28:09.412452456 (87h59m9.283535491s) [upload=76792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Dec 9 07:28:09.412455432 (87h59m9.283538477s) [upload=76792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 07:28:19.867794892 (87h59m19.738877927s) [upload=76792]: Done uploading segments... Dec 9 07:28:19.867843895 (87h59m19.73892694s) [upload=76792]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.013401436s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [............................X.XX.XX..X..................X...] duration: 3.501285568s trace: https://jaeger.dev.storj.io/trace/610b5afc07f67951 FAIL (2025-12-09 07:50:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.501352545s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 6.145196277s trace: https://jaeger.dev.storj.io/trace/5f10a1154ce06783 FAIL (2025-12-09 07:50:37 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.145219211s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X....................X.....] duration: 5.708213795s trace: https://jaeger.dev.storj.io/trace/7fe88be41ea8781e FAIL (2025-12-09 07:50:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng piece... Dec 9 07:45:33.502533729 (88h16m33.373616774s) [noise=true , address=77.90.31.11:4580 , piece_id=UDQZXQNUWDEILKNR42DLKTYVIJX3M7Q2YH3KOCBETUEBQZZ2WU4A, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.503057677 (88h16m33.374140712s) [noise=true , address=77.90.31.17:4797 , piece_id=XQINPPZJLK6HMR33M2SGC3EHJKL4XX7KZRWNM532PZHD3DEQ4M3Q, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.503561135 (88h16m33.37464417s) [noise=true , address=45.91.116.6:4987 , piece_id=JEQK2JKSRMQLGLNLVPBCPS6MCRYOJ7W7IEB4HBTBYFLIGO3CRXKQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.504037222 (88h16m33.375120257s) [noise=true , address=77.90.31.10:4546 , piece_id=SNWHQLYV5CBLDDT2BHM7IKY25CHDCJFTTT2APV2IN7RQQIVG7TZA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.504531994 (88h16m33.375615039s) [noise=true , address=77.90.31.21:4895 , piece_id=SW37FGLFYCZCFUVZL7BISFI3FQS2G2RXXYLD2IPOH75UE475HCTQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.505014954 (88h16m33.376098029s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.505576262 (88h16m33.376659307s) [noise=true , address=45.91.116.8:4071 , piece_id=INHPP6K7NOVPOV2G6CAUFSIKF2Q3KB5SOEE53MVU4MUDPMLSLC4A, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.506942875 (88h16m33.37802591s) [noise=true , address=45.91.116.11:4133 , piece_id=V72CQI43EBT2QWQKAC5GU2MSB7UKF7G5NIGMTLS7EOLOBGEG4YQQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.508063960 (88h16m33.379147005s) [noise=true , address=77.90.31.16:4749 , piece_id=B7JNUSR74QRRNXEFUYILTIMALV5DN75TX7P67VQ6TDGXLBLFDCGQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.508207313 (88h16m33.379290358s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.509568444 (88h16m33.38065149s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.511408599 (88h16m33.382491644s) [noise=true , address=77.90.31.19:4875 , piece_id=EA3XPU4W6DRDZMYWY2XW4IQ35DMCQEV7XQSNHJN33H24K7MJVOKA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.513502646 (88h16m33.384585681s) [noise=true , address=77.90.31.14:4683 , piece_id=FAS45YECJULRHU63VG4NFL3NPFCZPQW7IBELFWGVGMIE7YQS3J6Q, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.515541418 (88h16m33.386624484s) [noise=true , address=145.79.227.23:4565 , piece_id=PRBWTR7SVNCCCP5ERNUPB6TJOPXSPYFSZJIOL3RLPC4KRVORYWXQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.517269299 (88h16m33.388352334s) [noise=true , address=145.79.227.52:4452 , piece_id=X2RBQGFCIHR4JAJTJJ3FX4CA6PQIQUA2YHYJUANV7JIU5PKAVNLA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.518893461 (88h16m33.389976496s) [noise=true , address=77.90.31.15:4732 , piece_id=HYM7NBL4TFURODTBMWBJ7J23DON4WNYFJQ4Q3WNEBN7KCZYV3ORA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:34.970483840 (88h16m34.841566885s) [upload=77044]: Next returned nil segment Dec 9 07:45:34.970497998 (88h16m34.841581043s) [upload=77044]: Waiting for error group managing segments... Dec 9 07:45:35.375513876 (88h16m35.246596921s) [noise=true , address=145.79.227.23:4565 , piece_id=PRBWTR7SVNCCCP5ERNUPB6TJOPXSPYFSZJIOL3RLPC4KRVORYWXQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.378851702 (88h16m35.249934747s) [noise=true , address=145.79.227.52:4452 , piece_id=X2RBQGFCIHR4JAJTJJ3FX4CA6PQIQUA2YHYJUANV7JIU5PKAVNLA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.452093509 (88h16m35.323176544s) [noise=true , address=77.90.31.21:4895 , piece_id=SW37FGLFYCZCFUVZL7BISFI3FQS2G2RXXYLD2IPOH75UE475HCTQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.484823964 (88h16m35.355906999s) [noise=true , address=77.90.31.19:4875 , piece_id=EA3XPU4W6DRDZMYWY2XW4IQ35DMCQEV7XQSNHJN33H24K7MJVOKA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.501356315 (88h16m35.37243935s) [noise=true , address=77.90.31.16:4749 , piece_id=B7JNUSR74QRRNXEFUYILTIMALV5DN75TX7P67VQ6TDGXLBLFDCGQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.506575192 (88h16m35.377658238s) [noise=true , address=77.90.31.14:4683 , piece_id=FAS45YECJULRHU63VG4NFL3NPFCZPQW7IBELFWGVGMIE7YQS3J6Q, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.511021099 (88h16m35.382104134s) [noise=true , address=77.90.31.17:4797 , piece_id=XQINPPZJLK6HMR33M2SGC3EHJKL4XX7KZRWNM532PZHD3DEQ4M3Q, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.519588893 (88h16m35.390671938s) [noise=true , address=77.90.31.11:4580 , piece_id=UDQZXQNUWDEILKNR42DLKTYVIJX3M7Q2YH3KOCBETUEBQZZ2WU4A, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.540564915 (88h16m35.41164795s) [noise=true , address=45.91.116.11:4133 , piece_id=V72CQI43EBT2QWQKAC5GU2MSB7UKF7G5NIGMTLS7EOLOBGEG4YQQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.555394233 (88h16m35.426477278s) [noise=true , address=77.90.31.10:4546 , piece_id=SNWHQLYV5CBLDDT2BHM7IKY25CHDCJFTTT2APV2IN7RQQIVG7TZA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.565337939 (88h16m35.436420984s) [noise=true , address=45.91.116.8:4071 , piece_id=INHPP6K7NOVPOV2G6CAUFSIKF2Q3KB5SOEE53MVU4MUDPMLSLC4A, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.576688943 (88h16m35.447771978s) [noise=true , address=77.90.31.15:4732 , piece_id=HYM7NBL4TFURODTBMWBJ7J23DON4WNYFJQ4Q3WNEBN7KCZYV3ORA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.589009313 (88h16m35.460092368s) [noise=true , address=45.91.116.13:4181 , piece_id=X6IEEUKM5GDXZD2URW6YAPSN4DXWOCD2NK56K5PFFAYEHL2RTJUQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.619301786 (88h16m35.490384821s) [noise=true , address=45.91.116.6:4987 , piece_id=JEQK2JKSRMQLGLNLVPBCPS6MCRYOJ7W7IEB4HBTBYFLIGO3CRXKQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.619323929 (88h16m35.490406964s) [seg_pos={0 0}, upload=77044]: Segment reached optimal threshold of 14 pieces. Dec 9 07:45:35.619554758 (88h16m35.490637793s) [noise=true , address=45.91.116.12:4231 , piece_id=HZ7JTNUJBWREEUHR3A6D5GG5ENKCA4MXAHCCNMI4MU6ZHKZUC6MQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NmgZdLWKjrDMmNcgmnzH475akX4WtA9zGTLxZgVW7JpaecBjq): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619564476 (88h16m35.490647531s) [noise=true , address=45.91.116.12:4231 , piece_id=HZ7JTNUJBWREEUHR3A6D5GG5ENKCA4MXAHCCNMI4MU6ZHKZUC6MQ, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619577912 (88h16m35.490660947s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619583001 (88h16m35.490666037s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619751342 (88h16m35.490834377s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127KQ2X6VXddEeFRMau7MVGwtTsUmFsm8hTYDR2A6B4A1oVKfyo): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619757253 (88h16m35.490840298s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619853827 (88h16m35.490936862s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection Dec 9 07:45:35.619862484 (88h16m35.490945519s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619883484 (88h16m35.490966519s) [seg_pos={0 0}, upload=77044]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 07:45:35.619956242 (88h16m35.491039288s) [upload=77044]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 07:45:35.619961382 (88h16m35.491044417s) [upload=77044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 07:47:56.102293256 (88h18m55.973376291s) [upload=77044]: Done uploading segments... Dec 9 07:47:56.102346187 (88h18m55.973429222s) [upload=77044]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.708245076s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X...................X...................................] duration: 5.828620334s trace: https://jaeger.dev.storj.io/trace/59c049b2e292a646 FAIL (2025-12-09 07:50:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.82865036s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.923226746s trace: https://jaeger.dev.storj.io/trace/55634d9c1d55ff67 FAIL (2025-12-09 07:50:20 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.923248809s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.164940108s trace: https://jaeger.dev.storj.io/trace/50f433cc1e60731c FAIL (2025-12-09 07:50:14 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.164977779s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 5.767426068s trace: https://jaeger.dev.storj.io/trace/5f10a1154ce06783 FAIL (2025-12-09 07:50:12 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.767857039s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X....................X....] duration: 6.15440904s trace: https://jaeger.dev.storj.io/trace/7fe88be41ea8781e FAIL (2025-12-09 07:50:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng piece... Dec 9 07:45:33.502533729 (88h16m33.373616774s) [noise=true , address=77.90.31.11:4580 , piece_id=UDQZXQNUWDEILKNR42DLKTYVIJX3M7Q2YH3KOCBETUEBQZZ2WU4A, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.503057677 (88h16m33.374140712s) [noise=true , address=77.90.31.17:4797 , piece_id=XQINPPZJLK6HMR33M2SGC3EHJKL4XX7KZRWNM532PZHD3DEQ4M3Q, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.503561135 (88h16m33.37464417s) [noise=true , address=45.91.116.6:4987 , piece_id=JEQK2JKSRMQLGLNLVPBCPS6MCRYOJ7W7IEB4HBTBYFLIGO3CRXKQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.504037222 (88h16m33.375120257s) [noise=true , address=77.90.31.10:4546 , piece_id=SNWHQLYV5CBLDDT2BHM7IKY25CHDCJFTTT2APV2IN7RQQIVG7TZA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.504531994 (88h16m33.375615039s) [noise=true , address=77.90.31.21:4895 , piece_id=SW37FGLFYCZCFUVZL7BISFI3FQS2G2RXXYLD2IPOH75UE475HCTQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.505014954 (88h16m33.376098029s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.505576262 (88h16m33.376659307s) [noise=true , address=45.91.116.8:4071 , piece_id=INHPP6K7NOVPOV2G6CAUFSIKF2Q3KB5SOEE53MVU4MUDPMLSLC4A, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.506942875 (88h16m33.37802591s) [noise=true , address=45.91.116.11:4133 , piece_id=V72CQI43EBT2QWQKAC5GU2MSB7UKF7G5NIGMTLS7EOLOBGEG4YQQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.508063960 (88h16m33.379147005s) [noise=true , address=77.90.31.16:4749 , piece_id=B7JNUSR74QRRNXEFUYILTIMALV5DN75TX7P67VQ6TDGXLBLFDCGQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.508207313 (88h16m33.379290358s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.509568444 (88h16m33.38065149s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.511408599 (88h16m33.382491644s) [noise=true , address=77.90.31.19:4875 , piece_id=EA3XPU4W6DRDZMYWY2XW4IQ35DMCQEV7XQSNHJN33H24K7MJVOKA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.513502646 (88h16m33.384585681s) [noise=true , address=77.90.31.14:4683 , piece_id=FAS45YECJULRHU63VG4NFL3NPFCZPQW7IBELFWGVGMIE7YQS3J6Q, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.515541418 (88h16m33.386624484s) [noise=true , address=145.79.227.23:4565 , piece_id=PRBWTR7SVNCCCP5ERNUPB6TJOPXSPYFSZJIOL3RLPC4KRVORYWXQ, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.517269299 (88h16m33.388352334s) [noise=true , address=145.79.227.52:4452 , piece_id=X2RBQGFCIHR4JAJTJJ3FX4CA6PQIQUA2YHYJUANV7JIU5PKAVNLA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:33.518893461 (88h16m33.389976496s) [noise=true , address=77.90.31.15:4732 , piece_id=HYM7NBL4TFURODTBMWBJ7J23DON4WNYFJQ4Q3WNEBN7KCZYV3ORA, seg_pos={0 0}, upload=77044]: Uploading piece... Dec 9 07:45:34.970483840 (88h16m34.841566885s) [upload=77044]: Next returned nil segment Dec 9 07:45:34.970497998 (88h16m34.841581043s) [upload=77044]: Waiting for error group managing segments... Dec 9 07:45:35.375513876 (88h16m35.246596921s) [noise=true , address=145.79.227.23:4565 , piece_id=PRBWTR7SVNCCCP5ERNUPB6TJOPXSPYFSZJIOL3RLPC4KRVORYWXQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.378851702 (88h16m35.249934747s) [noise=true , address=145.79.227.52:4452 , piece_id=X2RBQGFCIHR4JAJTJJ3FX4CA6PQIQUA2YHYJUANV7JIU5PKAVNLA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.452093509 (88h16m35.323176544s) [noise=true , address=77.90.31.21:4895 , piece_id=SW37FGLFYCZCFUVZL7BISFI3FQS2G2RXXYLD2IPOH75UE475HCTQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.484823964 (88h16m35.355906999s) [noise=true , address=77.90.31.19:4875 , piece_id=EA3XPU4W6DRDZMYWY2XW4IQ35DMCQEV7XQSNHJN33H24K7MJVOKA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.501356315 (88h16m35.37243935s) [noise=true , address=77.90.31.16:4749 , piece_id=B7JNUSR74QRRNXEFUYILTIMALV5DN75TX7P67VQ6TDGXLBLFDCGQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.506575192 (88h16m35.377658238s) [noise=true , address=77.90.31.14:4683 , piece_id=FAS45YECJULRHU63VG4NFL3NPFCZPQW7IBELFWGVGMIE7YQS3J6Q, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.511021099 (88h16m35.382104134s) [noise=true , address=77.90.31.17:4797 , piece_id=XQINPPZJLK6HMR33M2SGC3EHJKL4XX7KZRWNM532PZHD3DEQ4M3Q, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.519588893 (88h16m35.390671938s) [noise=true , address=77.90.31.11:4580 , piece_id=UDQZXQNUWDEILKNR42DLKTYVIJX3M7Q2YH3KOCBETUEBQZZ2WU4A, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.540564915 (88h16m35.41164795s) [noise=true , address=45.91.116.11:4133 , piece_id=V72CQI43EBT2QWQKAC5GU2MSB7UKF7G5NIGMTLS7EOLOBGEG4YQQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.555394233 (88h16m35.426477278s) [noise=true , address=77.90.31.10:4546 , piece_id=SNWHQLYV5CBLDDT2BHM7IKY25CHDCJFTTT2APV2IN7RQQIVG7TZA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.565337939 (88h16m35.436420984s) [noise=true , address=45.91.116.8:4071 , piece_id=INHPP6K7NOVPOV2G6CAUFSIKF2Q3KB5SOEE53MVU4MUDPMLSLC4A, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.576688943 (88h16m35.447771978s) [noise=true , address=77.90.31.15:4732 , piece_id=HYM7NBL4TFURODTBMWBJ7J23DON4WNYFJQ4Q3WNEBN7KCZYV3ORA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.589009313 (88h16m35.460092368s) [noise=true , address=45.91.116.13:4181 , piece_id=X6IEEUKM5GDXZD2URW6YAPSN4DXWOCD2NK56K5PFFAYEHL2RTJUQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.619301786 (88h16m35.490384821s) [noise=true , address=45.91.116.6:4987 , piece_id=JEQK2JKSRMQLGLNLVPBCPS6MCRYOJ7W7IEB4HBTBYFLIGO3CRXKQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: <nil> Dec 9 07:45:35.619323929 (88h16m35.490406964s) [seg_pos={0 0}, upload=77044]: Segment reached optimal threshold of 14 pieces. Dec 9 07:45:35.619554758 (88h16m35.490637793s) [noise=true , address=45.91.116.12:4231 , piece_id=HZ7JTNUJBWREEUHR3A6D5GG5ENKCA4MXAHCCNMI4MU6ZHKZUC6MQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NmgZdLWKjrDMmNcgmnzH475akX4WtA9zGTLxZgVW7JpaecBjq): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619564476 (88h16m35.490647531s) [noise=true , address=45.91.116.12:4231 , piece_id=HZ7JTNUJBWREEUHR3A6D5GG5ENKCA4MXAHCCNMI4MU6ZHKZUC6MQ, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619577912 (88h16m35.490660947s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619583001 (88h16m35.490666037s) [noise=true , address=145.79.227.30:4796 , piece_id=I25OEVIWGQMZHFZMJMP7H5BMET6GX72RAEBEKV2ZSGQHQGP6QNTA, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619751342 (88h16m35.490834377s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127KQ2X6VXddEeFRMau7MVGwtTsUmFsm8hTYDR2A6B4A1oVKfyo): protocol: expected piece hash; context canceled; EOF Dec 9 07:45:35.619757253 (88h16m35.490840298s) [noise=true , address=77.90.31.18:4827 , piece_id=UK3RJZ5DHNVWCMTUWGTAEJJCY7YXK7EWSQKGEQYV4JYFZ6AV3FEQ, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619853827 (88h16m35.490936862s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34936->64.49.13.169:4707: use of closed network connection Dec 9 07:45:35.619862484 (88h16m35.490945519s) [noise=true , address=64.49.13.169:4707 , piece_id=43LJJAVA7GKYJEUEYARA5BD7VYVAC3HOH3PFOWFTJEYZXDANCAVA, seg_pos={0 0}, upload=77044]: Long tail cancellation due to optimal threshold Dec 9 07:45:35.619883484 (88h16m35.490966519s) [seg_pos={0 0}, upload=77044]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 07:45:35.619956242 (88h16m35.491039288s) [upload=77044]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 07:45:35.619961382 (88h16m35.491044417s) [upload=77044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 07:47:56.102293256 (88h18m55.973376291s) [upload=77044]: Done uploading segments... Dec 9 07:47:56.102346187 (88h18m55.973429222s) [upload=77044]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.154443366s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.022103969s trace: https://jaeger.dev.storj.io/trace/55634d9c1d55ff67 FAIL (2025-12-09 07:50:05 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.022149195s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 3 out of 60 is failed (threshold=3) [............................XX..X...........................] duration: 696.8782ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/43834d9fac36e837 FAIL (2025-12-09 07:50:05 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 696.906455ms core satellite us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [............................XXX.............................] duration: 2.150762059s trace: https://jaeger.dev.storj.io/trace/178b4cba623ae119 FAIL (2025-12-09 07:50:05 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.150801894s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................XXX.X...............................] duration: 5.646551259s trace: https://jaeger.dev.storj.io/trace/34e772a771ff36b2 FAIL (2025-12-09 07:49:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== :29:22.029442363 (88h0m21.900525398s) [noise=true , address=77.90.31.10:4519 , piece_id=PLAP7Q4BYYOUCI53OQKNR74TS4XCPEI6GTH5CETYOWWCEPIDCTEA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.029471198 (88h0m21.900554233s) [noise=true , address=145.79.227.20:4438 , piece_id=NC5ZFPAH575LULPHREYIQXSW44HHPJEUAVCM2RZYOACUWYOD6D6A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.030555563 (88h0m21.901638608s) [noise=true , address=145.79.227.26:4654 , piece_id=LH5BDOW3HG7P7VSLJ3EUBPLOPK3DPAXSJB5RMA5KFEMDQ3R52W3A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.031156617 (88h0m21.902239662s) [noise=true , address=77.90.31.10:4520 , piece_id=5JGFSZCUORYTBSH5IEYGLGW3GJ7IIKZZBUNG7ZYI2DWKCAC7FN4A, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.031594381 (88h0m21.902677416s) [noise=true , address=145.79.227.39:4014 , piece_id=QSO5DWCCBHO7WP2XJO2Y7RP3WMSJPIEZZWNKYNB6CHJNXXKTXUQQ, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032034670 (88h0m21.903117715s) [noise=true , address=145.79.227.2:4114 , piece_id=KNMNK52ESYIBOSEF4HTDJOFJSOXHXQWRUI5URRQ6WRQOFGWM47SA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032167963 (88h0m21.903250998s) [noise=true , address=145.79.227.40:4049 , piece_id=OCQVF3EKPS273EYHIPTFDMIBPQF63OITQVS6RDUNLCOCRNWBS3EQ, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:22.032800087 (88h0m21.903883132s) [noise=true , address=145.79.227.32:4834 , piece_id=LZGJJKA7LBTTBN26ZKVJKIMIQJDOXZVEIEUIL4Y73RPOFGJZOAOA, seg_pos={0 0}, upload=76810]: Uploading piece... Dec 9 07:29:23.836496806 (88h0m23.707579841s) [upload=76810]: Next returned nil segment Dec 9 07:29:23.836512486 (88h0m23.707595531s) [upload=76810]: Waiting for error group managing segments... Dec 9 07:29:24.124325638 (88h0m23.995408683s) [noise=true , address=77.90.31.10:4519 , piece_id=PLAP7Q4BYYOUCI53OQKNR74TS4XCPEI6GTH5CETYOWWCEPIDCTEA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.114522896 (88h0m24.985605941s) [noise=true , address=145.79.227.21:4475 , piece_id=ED4VS3YEIFPJGGKW36BJKVLT3EXGVOFG3SD23LEPPFIAOOKHUD4Q, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.377083973 (88h0m25.248167008s) [noise=true , address=145.79.227.41:4086 , piece_id=EZUBNMEUWTNJBXJBWJYXFJQEPFFVQOUHZAWVJW6NCWKGRKAS2PSA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:25.480466423 (88h0m25.351549468s) [noise=true , address=145.79.227.42:4121 , piece_id=7I2TXNXZM2KM5TEDDF7Q5DDUSBAPBE3WWFTTJFGNM5JVVCTQNU5Q, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:26.767037853 (88h0m26.638120908s) [noise=true , address=77.90.31.10:4520 , piece_id=5JGFSZCUORYTBSH5IEYGLGW3GJ7IIKZZBUNG7ZYI2DWKCAC7FN4A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: <nil> Dec 9 07:29:49.689337786 (88h0m49.560420841s) [noise=true , address=145.79.227.40:4049 , piece_id=OCQVF3EKPS273EYHIPTFDMIBPQF63OITQVS6RDUNLCOCRNWBS3EQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection; write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35260->145.79.227.40:4049: use of closed network connection Dec 9 07:29:49.689500304 (88h0m49.56058335s) [noise=true , address=145.79.227.34:4907 , piece_id=UG6M3T655YGW4OGBYEVOLQVR7U5XWXPDDNXBZAPTHYP2DYWDRRYA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS, address:145.79.227.34:4907): protocol: write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection; write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48142->145.79.227.34:4907: use of closed network connection Dec 9 07:29:49.689541303 (88h0m49.560624348s) [noise=true , address=145.79.227.33:4870 , piece_id=6Q5XSDCJCDRWHPO7JWCDNPQM5JN2ESDCRXWY5V23AGA6Q4C5Y7HQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt, address:145.79.227.33:4870): protocol: write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection; write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48964->145.79.227.33:4870: use of closed network connection Dec 9 07:29:49.689654439 (88h0m49.560737474s) [noise=true , address=145.79.227.29:4943 , piece_id=FQ2FNNQR6QBWTJOIQBIOU7BSWIA6ZZMDWC4DE42NUNR5Z7ZDGCGQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection; write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48622->145.79.227.29:4943: use of closed network connection Dec 9 07:29:49.689700015 (88h0m49.56078306s) [noise=true , address=145.79.227.42:4122 , piece_id=5WKTAWBKWCZVU3YX4RSF4FQHQABNCZUHTXMTK3AIHUAGYDQXQYVQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:124d52DhtbtwipVpZuWdpFTQAttR18hy1K1BbokNRX4R1bLtzTi, address:145.79.227.42:4122): protocol: write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection; write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53248->145.79.227.42:4122: use of closed network connection Dec 9 07:29:49.689861212 (88h0m49.560944247s) [noise=true , address=145.79.227.39:4014 , piece_id=QSO5DWCCBHO7WP2XJO2Y7RP3WMSJPIEZZWNKYNB6CHJNXXKTXUQQ, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection; write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60706->145.79.227.39:4014: use of closed network connection Dec 9 07:29:49.689914644 (88h0m49.560997689s) [noise=true , address=145.79.227.28:4725 , piece_id=AGDXVUHDFM7DC2HNW3YUKREAROODLZBIVUMTFYBY3CXYQ7HSYGWA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34086->145.79.227.28:4725: use of closed network connection Dec 9 07:29:49.690003323 (88h0m49.561086368s) [noise=true , address=145.79.227.32:4834 , piece_id=LZGJJKA7LBTTBN26ZKVJKIMIQJDOXZVEIEUIL4Y73RPOFGJZOAOA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection; write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43002->145.79.227.32:4834: use of closed network connection Dec 9 07:29:49.690090949 (88h0m49.561173994s) [noise=true , address=145.79.227.26:4654 , piece_id=LH5BDOW3HG7P7VSLJ3EUBPLOPK3DPAXSJB5RMA5KFEMDQ3R52W3A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34578->145.79.227.26:4654: use of closed network connection Dec 9 07:29:49.690262546 (88h0m49.561345591s) [noise=true , address=145.79.227.2:4114 , piece_id=KNMNK52ESYIBOSEF4HTDJOFJSOXHXQWRUI5URRQ6WRQOFGWM47SA, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55674->145.79.227.2:4114: use of closed network connection Dec 9 07:29:49.690582726 (88h0m49.561665772s) [noise=true , address=145.79.227.20:4438 , piece_id=NC5ZFPAH575LULPHREYIQXSW44HHPJEUAVCM2RZYOACUWYOD6D6A, seg_pos={0 0}, upload=76810]: Done uploading piece. err: ecclient: upload failed (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT, address:145.79.227.20:4438): protocol: read tcp 188.166.225.170:50466->145.79.227.20:4438: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 07:29:49.690631769 (88h0m49.561714805s) [seg_pos={0 0}, upload=76810]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Dec 9 07:29:50.737500571 (88h0m50.608583616s) [upload=76810]: Done uploading segments... Dec 9 07:29:50.737550175 (88h0m50.60863321s) [upload=76810]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.646581557s core uplink us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 450.85762ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt trace: https://jaeger.dev.storj.io/trace/51a280a8635ce0b9 FAIL (2025-12-09 07:49:58 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 450.95642ms edge linksharing us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............................XXX............................] duration: 2.160926665s trace: https://jaeger.dev.storj.io/trace/178b4cba623ae119 FAIL (2025-12-09 07:49:46 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.160949971s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)