Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 864863 reports (Page 11688 of 17298)
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................]
duration:
702.817179ms
remote:
"tcp:35.212.10.183:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/3abf551f64a5a0f9
FAIL (2025-12-08 18:34:45 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 702.840675ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
6 out of 60 is failed (threshold=6) [..........X..X...............X............X...X......X......]
duration:
3.484411543s
trace:
https://jaeger.dev.storj.io/trace/1c7f30f7e7627ffd
FAIL (2025-12-08 18:34:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.484436831s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...........X........X.......................................]
duration:
5.22944233s
trace:
https://jaeger.dev.storj.io/trace/355abf85f96191de
FAIL (2025-12-08 18:34:41 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.229465563s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
2 out of 60 is failed (threshold=3) [.....................X........................X.............]
duration:
5.912680627s
trace:
https://jaeger.dev.storj.io/trace/323c8ad1ea235936
FAIL (2025-12-08 18:34:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.91270882s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................]
duration:
1.994865679s
trace:
https://jaeger.dev.storj.io/trace/54ea49bbbba7a774
FAIL (2025-12-08 18:34:36 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 1.994886499s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................]
duration:
698.25323ms
remote:
"tcp:35.212.10.183:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/3abf551f64a5a0f9
FAIL (2025-12-08 18:33:57 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 698.283517ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
6 out of 60 is failed (threshold=6) [...........X..X...............X............X...X......X.....]
duration:
3.466981715s
trace:
https://jaeger.dev.storj.io/trace/1c7f30f7e7627ffd
FAIL (2025-12-08 18:33:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.467022052s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................]
duration:
5.270306392s
trace:
https://jaeger.dev.storj.io/trace/2bd986c05c15b190
FAIL (2025-12-08 18:33:55 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
CDZRW6LEEY46BBQFTFZE5DYUXXXHUTIN3CVN63K65VQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.842730397 (74h24m40.713813432s) [noise=true , address=185.167.204.205:4032 , piece_id=NF4LIZE2CVGDLBP5O4TFDTJBINY76W2L6ARPSVV5AJXBREKTQS7Q, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.843290634 (74h24m40.714373669s) [noise=true , address=185.167.204.206:4064 , piece_id=2WY55HXXR5LPAR6XSD4GYYYA5BV3A2DPQ5LCPOL3UAWM4IU4ZIJA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.845386525 (74h24m40.716469571s) [noise=true , address=81.171.14.70:4029    , piece_id=3QQWIZOHYA4MRXNV37TOPRD3WWE3V45D2M474ADRLUHB47FLFUIQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.846892383 (74h24m40.717975428s) [noise=true , address=89.149.201.147:4010  , piece_id=XGAAAQTHKLOR3YLIQW3PKYQGKLES5ZHUHYMDHXFVQWELBQCLYFOQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.848772573 (74h24m40.719855618s) [noise=true , address=185.167.204.206:4049 , piece_id=7IYDBS2GFRUWYC5O62HTXPOL5IPXQ7NZPE4WHDBN5AEAEPECH2VA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.850191745 (74h24m40.72127478s) [noise=true , address=185.167.204.205:4017 , piece_id=2TUWBAIWW6E7GORGD6MBNQFIXUHTD2DTCOVBRJCJW4BFTOPLONCA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.851609155 (74h24m40.7226922s) [noise=true , address=89.149.201.147:4021  , piece_id=NX5FUM4724BSTFCEKRLLARMVFV7BFIXKT66XZ6RLIZTPMPWXN2AA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.852832574 (74h24m40.723915619s) [noise=true , address=81.171.14.70:4036    , piece_id=XBKXRAUGGAQVV62WSVGNPKXILMXBTIN2VWIAHJM37XLWLUSXWHNA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.854147148 (74h24m40.725230203s) [noise=true , address=81.171.14.70:4035    , piece_id=ULGXVDR7SYRD5FW5B52QXGXQRYUPU3FT4QFSCHWQDXRJ2JM5725Q, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:41.670064128 (74h24m41.541147173s) [upload=65114]: Next returned nil segment
Dec  8 17:53:41.670145664 (74h24m41.541228709s) [upload=65114]: Waiting for error group managing segments...
Dec  8 17:53:42.057773064 (74h24m41.928856099s) [noise=true , address=185.167.204.206:4065 , piece_id=7GQAOZEZ6OJKW3UK7U3NXYZM7VXGPHKW6UKHSTEUKJDNRZAYUA4A, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.058200909 (74h24m41.929283954s) [noise=true , address=185.167.204.206:4045 , piece_id=6O2I5IIAPPQFG66OPIOVY2GOHO4C5CM2UXMJHGY45WGQVZK3JGPQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.058523583 (74h24m41.929606628s) [noise=true , address=185.167.204.205:4034 , piece_id=6JYH3N55MJSMXE6VNGSZK7DZX7KUV7CPELNHIMPSFOXG4U2NXRTQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.060741817 (74h24m41.931824863s) [noise=true , address=185.167.204.205:4025 , piece_id=2MDDUA2MUJ6MXICI5H5O4KPAIRJUL7QLBIC4L6MIEKNDYQIO2EJQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.062816829 (74h24m41.933899864s) [noise=true , address=185.167.204.206:4072 , piece_id=X5XLPXKRFRIEQ32PZKY23AS4CFBXSC5NVCJF24IY2LQH4KLGG6RA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.064906518 (74h24m41.935989563s) [noise=true , address=185.167.204.206:4059 , piece_id=KPZWLMWMZA7GCT6JWIMSMLGUW6UVJ3FPWJKMSHP72C42TH5Z4ITA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.065603074 (74h24m41.93668611s) [noise=true , address=185.167.204.206:4047 , piece_id=4G5HDKWP26ER64KVLUXJ2IMQJUXMQ4YLNBYQAMY7MQMGE2PSREEA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.066790916 (74h24m41.937873951s) [noise=true , address=185.167.204.205:4017 , piece_id=2TUWBAIWW6E7GORGD6MBNQFIXUHTD2DTCOVBRJCJW4BFTOPLONCA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.075608666 (74h24m41.946691701s) [noise=true , address=185.167.204.206:4064 , piece_id=2WY55HXXR5LPAR6XSD4GYYYA5BV3A2DPQ5LCPOL3UAWM4IU4ZIJA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.076342644 (74h24m41.947425679s) [noise=true , address=185.167.204.205:4036 , piece_id=FKH55HUCJTNREM6PBSFGWURF5HIGWAM3U6OPQQVTRGEUA6K7JM2Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.077612843 (74h24m41.948695888s) [noise=true , address=185.167.204.205:4032 , piece_id=NF4LIZE2CVGDLBP5O4TFDTJBINY76W2L6ARPSVV5AJXBREKTQS7Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.078994745 (74h24m41.95007778s) [noise=true , address=185.167.204.206:4046 , piece_id=SDP7J6VHYTCJSKOHR4AXONUYML7VNYQIVATGTPENHWS4F4CO3A4A, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.082062506 (74h24m41.953145541s) [noise=true , address=185.167.204.206:4070 , piece_id=5ETK46YMRH46TMYDEYR4CPV3XDF3FS5MR5RSOYPZKD33EXMS5TQQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.096426879 (74h24m41.967509924s) [noise=true , address=185.167.204.206:4049 , piece_id=7IYDBS2GFRUWYC5O62HTXPOL5IPXQ7NZPE4WHDBN5AEAEPECH2VA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.766933610 (74h24m42.638016655s) [noise=true , address=81.171.14.70:4031    , piece_id=S4T3ORC533X2OW2Y3BQDAFSYXBTD3THLREQMMJ2G7QPM3UDGHRCQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.860649744 (74h24m42.731732789s) [noise=true , address=81.171.14.70:4029    , piece_id=3QQWIZOHYA4MRXNV37TOPRD3WWE3V45D2M474ADRLUHB47FLFUIQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.874184617 (74h24m42.745267652s) [noise=true , address=81.171.14.70:4035    , piece_id=ULGXVDR7SYRD5FW5B52QXGXQRYUPU3FT4QFSCHWQDXRJ2JM5725Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.254322126 (74h24m44.125405172s) [noise=true , address=89.149.201.147:4010  , piece_id=XGAAAQTHKLOR3YLIQW3PKYQGKLES5ZHUHYMDHXFVQWELBQCLYFOQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.671496219 (74h24m44.542579254s) [noise=true , address=81.171.14.70:4025    , piece_id=YX5EYUXYW4ZY3L6CH673H5BSHKUEKTUGGKVSFVENUVJBF3ONZUKA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.750460513 (74h24m44.621543558s) [noise=true , address=81.171.14.70:4033    , piece_id=LSIJ6NCTBALZH2QRPFFTLOXKKWF7J2E3NL2GIYXCDF2RC5KMNBPA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.792368875 (74h24m44.66345192s) [noise=true , address=81.171.14.70:4027    , piece_id=X53DOOCXSAHI64TN5GNSEOHBJK7GL6TLL6W7NJM5KQIVW3ITZ2BA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:45.851581290 (74h24m45.722664326s) [noise=true , address=81.171.14.70:4036    , piece_id=XBKXRAUGGAQVV62WSVGNPKXILMXBTIN2VWIAHJM37XLWLUSXWHNA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:46.587984219 (74h24m46.459067264s) [noise=true , address=89.149.201.147:4022  , piece_id=ZEX4GP43WCDZRW6LEEY46BBQFTFZE5DYUXXXHUTIN3CVN63K65VQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:46.808512099 (74h24m46.679595144s) [noise=true , address=81.171.14.70:4037    , piece_id=MHNTZCXG6COLI7J45SSYK6SZDVSSEBXOFFY7KEGP7NJUDEVLX7NA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:47.152614810 (74h24m47.023697845s) [noise=true , address=81.171.14.70:4038    , piece_id=VFW2FCTINXQNF3AFAFQW5AYWC2EZ2P52ERVUFU5DMH7672AWAQTA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:49.686376464 (74h24m49.557459499s) [noise=true , address=89.149.201.147:4021  , piece_id=NX5FUM4724BSTFCEKRLLARMVFV7BFIXKT66XZ6RLIZTPMPWXN2AA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:49.686397555 (74h24m49.55748059s) [seg_pos={0 0}, upload=65114]: Segment reached optimal threshold of 26 pieces.
Dec  8 17:53:49.686927403 (74h24m49.558010438s) [noise=true , address=89.149.201.147:4009  , piece_id=7SFNKC45B4BXNTU6IWIETH6P2JXVCCSO2ICFUBDFATHIU3NC4NOA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F): protocol: read tcp 188.166.225.170:60652->89.149.201.147:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec  8 17:53:49.687035098 (74h24m49.558118143s) [noise=true , address=89.149.201.147:4009  , piece_id=7SFNKC45B4BXNTU6IWIETH6P2JXVCCSO2ICFUBDFATHIU3NC4NOA, seg_pos={0 0}, upload=65114]: Long tail cancellation due to optimal threshold
Dec  8 17:53:49.687151320 (74h24m49.558234365s) [noise=true , address=89.149.201.147:4001  , piece_id=4HDC333BXRRPHPLSYQIKPP5UW2A26C776YDY2GFTQKSFI76SSLGA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: read tcp 188.166.225.170:58886->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec  8 17:53:49.687225261 (74h24m49.558308326s) [noise=true , address=89.149.201.147:4001  , piece_id=4HDC333BXRRPHPLSYQIKPP5UW2A26C776YDY2GFTQKSFI76SSLGA, seg_pos={0 0}, upload=65114]: Long tail cancellation due to optimal threshold
Dec  8 17:53:49.687301826 (74h24m49.558384872s) [seg_pos={0 0}, upload=65114]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec  8 17:53:49.687437044 (74h24m49.558520089s) [upload=65114]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec  8 17:53:49.687467202 (74h24m49.558550247s) [upload=65114]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec  8 18:01:19.776524628 (74h32m19.647607673s) [upload=65114]: Done uploading segments...
Dec  8 18:01:19.776620922 (74h32m19.647703967s) [upload=65114]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.270339614s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
2 out of 60 is failed (threshold=3) [......................X........................X............]
duration:
5.886229971s
trace:
https://jaeger.dev.storj.io/trace/323c8ad1ea235936
FAIL (2025-12-08 18:33:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.886279183s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................]
duration:
1.981289448s
trace:
https://jaeger.dev.storj.io/trace/54ea49bbbba7a774
FAIL (2025-12-08 18:33:50 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 1.981314095s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
467.100249ms
remote:
"tcp:35.207.121.91:443"
target:
https://eu1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/25680d6a319febe4
FAIL (2025-12-08 18:33:50 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 467.124937ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
3 out of 60 is failed (threshold=3) [.X...........................X.....................X........]
duration:
5.827234854s
trace:
https://jaeger.dev.storj.io/trace/3d74ee1c8d24fd99
FAIL (2025-12-08 18:33:45 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.827269158s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [............X........X......................................]
duration:
5.130855292s
trace:
https://jaeger.dev.storj.io/trace/355abf85f96191de
FAIL (2025-12-08 18:33:44 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.130885901s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
6 out of 60 is failed (threshold=6) [............X..X...............X............X...X......X....]
duration:
3.574446596s
trace:
https://jaeger.dev.storj.io/trace/1c7f30f7e7627ffd
FAIL (2025-12-08 18:33:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.574477615s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................]
duration:
8.793946696s
trace:
https://jaeger.dev.storj.io/trace/2bd986c05c15b190
FAIL (2025-12-08 18:33:34 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
CDZRW6LEEY46BBQFTFZE5DYUXXXHUTIN3CVN63K65VQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.842730397 (74h24m40.713813432s) [noise=true , address=185.167.204.205:4032 , piece_id=NF4LIZE2CVGDLBP5O4TFDTJBINY76W2L6ARPSVV5AJXBREKTQS7Q, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.843290634 (74h24m40.714373669s) [noise=true , address=185.167.204.206:4064 , piece_id=2WY55HXXR5LPAR6XSD4GYYYA5BV3A2DPQ5LCPOL3UAWM4IU4ZIJA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.845386525 (74h24m40.716469571s) [noise=true , address=81.171.14.70:4029    , piece_id=3QQWIZOHYA4MRXNV37TOPRD3WWE3V45D2M474ADRLUHB47FLFUIQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.846892383 (74h24m40.717975428s) [noise=true , address=89.149.201.147:4010  , piece_id=XGAAAQTHKLOR3YLIQW3PKYQGKLES5ZHUHYMDHXFVQWELBQCLYFOQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.848772573 (74h24m40.719855618s) [noise=true , address=185.167.204.206:4049 , piece_id=7IYDBS2GFRUWYC5O62HTXPOL5IPXQ7NZPE4WHDBN5AEAEPECH2VA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.850191745 (74h24m40.72127478s) [noise=true , address=185.167.204.205:4017 , piece_id=2TUWBAIWW6E7GORGD6MBNQFIXUHTD2DTCOVBRJCJW4BFTOPLONCA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.851609155 (74h24m40.7226922s) [noise=true , address=89.149.201.147:4021  , piece_id=NX5FUM4724BSTFCEKRLLARMVFV7BFIXKT66XZ6RLIZTPMPWXN2AA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.852832574 (74h24m40.723915619s) [noise=true , address=81.171.14.70:4036    , piece_id=XBKXRAUGGAQVV62WSVGNPKXILMXBTIN2VWIAHJM37XLWLUSXWHNA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.854147148 (74h24m40.725230203s) [noise=true , address=81.171.14.70:4035    , piece_id=ULGXVDR7SYRD5FW5B52QXGXQRYUPU3FT4QFSCHWQDXRJ2JM5725Q, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:41.670064128 (74h24m41.541147173s) [upload=65114]: Next returned nil segment
Dec  8 17:53:41.670145664 (74h24m41.541228709s) [upload=65114]: Waiting for error group managing segments...
Dec  8 17:53:42.057773064 (74h24m41.928856099s) [noise=true , address=185.167.204.206:4065 , piece_id=7GQAOZEZ6OJKW3UK7U3NXYZM7VXGPHKW6UKHSTEUKJDNRZAYUA4A, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.058200909 (74h24m41.929283954s) [noise=true , address=185.167.204.206:4045 , piece_id=6O2I5IIAPPQFG66OPIOVY2GOHO4C5CM2UXMJHGY45WGQVZK3JGPQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.058523583 (74h24m41.929606628s) [noise=true , address=185.167.204.205:4034 , piece_id=6JYH3N55MJSMXE6VNGSZK7DZX7KUV7CPELNHIMPSFOXG4U2NXRTQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.060741817 (74h24m41.931824863s) [noise=true , address=185.167.204.205:4025 , piece_id=2MDDUA2MUJ6MXICI5H5O4KPAIRJUL7QLBIC4L6MIEKNDYQIO2EJQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.062816829 (74h24m41.933899864s) [noise=true , address=185.167.204.206:4072 , piece_id=X5XLPXKRFRIEQ32PZKY23AS4CFBXSC5NVCJF24IY2LQH4KLGG6RA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.064906518 (74h24m41.935989563s) [noise=true , address=185.167.204.206:4059 , piece_id=KPZWLMWMZA7GCT6JWIMSMLGUW6UVJ3FPWJKMSHP72C42TH5Z4ITA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.065603074 (74h24m41.93668611s) [noise=true , address=185.167.204.206:4047 , piece_id=4G5HDKWP26ER64KVLUXJ2IMQJUXMQ4YLNBYQAMY7MQMGE2PSREEA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.066790916 (74h24m41.937873951s) [noise=true , address=185.167.204.205:4017 , piece_id=2TUWBAIWW6E7GORGD6MBNQFIXUHTD2DTCOVBRJCJW4BFTOPLONCA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.075608666 (74h24m41.946691701s) [noise=true , address=185.167.204.206:4064 , piece_id=2WY55HXXR5LPAR6XSD4GYYYA5BV3A2DPQ5LCPOL3UAWM4IU4ZIJA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.076342644 (74h24m41.947425679s) [noise=true , address=185.167.204.205:4036 , piece_id=FKH55HUCJTNREM6PBSFGWURF5HIGWAM3U6OPQQVTRGEUA6K7JM2Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.077612843 (74h24m41.948695888s) [noise=true , address=185.167.204.205:4032 , piece_id=NF4LIZE2CVGDLBP5O4TFDTJBINY76W2L6ARPSVV5AJXBREKTQS7Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.078994745 (74h24m41.95007778s) [noise=true , address=185.167.204.206:4046 , piece_id=SDP7J6VHYTCJSKOHR4AXONUYML7VNYQIVATGTPENHWS4F4CO3A4A, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.082062506 (74h24m41.953145541s) [noise=true , address=185.167.204.206:4070 , piece_id=5ETK46YMRH46TMYDEYR4CPV3XDF3FS5MR5RSOYPZKD33EXMS5TQQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.096426879 (74h24m41.967509924s) [noise=true , address=185.167.204.206:4049 , piece_id=7IYDBS2GFRUWYC5O62HTXPOL5IPXQ7NZPE4WHDBN5AEAEPECH2VA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.766933610 (74h24m42.638016655s) [noise=true , address=81.171.14.70:4031    , piece_id=S4T3ORC533X2OW2Y3BQDAFSYXBTD3THLREQMMJ2G7QPM3UDGHRCQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.860649744 (74h24m42.731732789s) [noise=true , address=81.171.14.70:4029    , piece_id=3QQWIZOHYA4MRXNV37TOPRD3WWE3V45D2M474ADRLUHB47FLFUIQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.874184617 (74h24m42.745267652s) [noise=true , address=81.171.14.70:4035    , piece_id=ULGXVDR7SYRD5FW5B52QXGXQRYUPU3FT4QFSCHWQDXRJ2JM5725Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.254322126 (74h24m44.125405172s) [noise=true , address=89.149.201.147:4010  , piece_id=XGAAAQTHKLOR3YLIQW3PKYQGKLES5ZHUHYMDHXFVQWELBQCLYFOQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.671496219 (74h24m44.542579254s) [noise=true , address=81.171.14.70:4025    , piece_id=YX5EYUXYW4ZY3L6CH673H5BSHKUEKTUGGKVSFVENUVJBF3ONZUKA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.750460513 (74h24m44.621543558s) [noise=true , address=81.171.14.70:4033    , piece_id=LSIJ6NCTBALZH2QRPFFTLOXKKWF7J2E3NL2GIYXCDF2RC5KMNBPA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.792368875 (74h24m44.66345192s) [noise=true , address=81.171.14.70:4027    , piece_id=X53DOOCXSAHI64TN5GNSEOHBJK7GL6TLL6W7NJM5KQIVW3ITZ2BA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:45.851581290 (74h24m45.722664326s) [noise=true , address=81.171.14.70:4036    , piece_id=XBKXRAUGGAQVV62WSVGNPKXILMXBTIN2VWIAHJM37XLWLUSXWHNA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:46.587984219 (74h24m46.459067264s) [noise=true , address=89.149.201.147:4022  , piece_id=ZEX4GP43WCDZRW6LEEY46BBQFTFZE5DYUXXXHUTIN3CVN63K65VQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:46.808512099 (74h24m46.679595144s) [noise=true , address=81.171.14.70:4037    , piece_id=MHNTZCXG6COLI7J45SSYK6SZDVSSEBXOFFY7KEGP7NJUDEVLX7NA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:47.152614810 (74h24m47.023697845s) [noise=true , address=81.171.14.70:4038    , piece_id=VFW2FCTINXQNF3AFAFQW5AYWC2EZ2P52ERVUFU5DMH7672AWAQTA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:49.686376464 (74h24m49.557459499s) [noise=true , address=89.149.201.147:4021  , piece_id=NX5FUM4724BSTFCEKRLLARMVFV7BFIXKT66XZ6RLIZTPMPWXN2AA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:49.686397555 (74h24m49.55748059s) [seg_pos={0 0}, upload=65114]: Segment reached optimal threshold of 26 pieces.
Dec  8 17:53:49.686927403 (74h24m49.558010438s) [noise=true , address=89.149.201.147:4009  , piece_id=7SFNKC45B4BXNTU6IWIETH6P2JXVCCSO2ICFUBDFATHIU3NC4NOA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F): protocol: read tcp 188.166.225.170:60652->89.149.201.147:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec  8 17:53:49.687035098 (74h24m49.558118143s) [noise=true , address=89.149.201.147:4009  , piece_id=7SFNKC45B4BXNTU6IWIETH6P2JXVCCSO2ICFUBDFATHIU3NC4NOA, seg_pos={0 0}, upload=65114]: Long tail cancellation due to optimal threshold
Dec  8 17:53:49.687151320 (74h24m49.558234365s) [noise=true , address=89.149.201.147:4001  , piece_id=4HDC333BXRRPHPLSYQIKPP5UW2A26C776YDY2GFTQKSFI76SSLGA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: read tcp 188.166.225.170:58886->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec  8 17:53:49.687225261 (74h24m49.558308326s) [noise=true , address=89.149.201.147:4001  , piece_id=4HDC333BXRRPHPLSYQIKPP5UW2A26C776YDY2GFTQKSFI76SSLGA, seg_pos={0 0}, upload=65114]: Long tail cancellation due to optimal threshold
Dec  8 17:53:49.687301826 (74h24m49.558384872s) [seg_pos={0 0}, upload=65114]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec  8 17:53:49.687437044 (74h24m49.558520089s) [upload=65114]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec  8 17:53:49.687467202 (74h24m49.558550247s) [upload=65114]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec  8 18:01:19.776524628 (74h32m19.647607673s) [upload=65114]: Done uploading segments...
Dec  8 18:01:19.776620922 (74h32m19.647703967s) [upload=65114]: Upload finished. err: metaclient: context deadline exceeded
Duration: 8.793973237s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................]
duration:
2.006101966s
trace:
https://jaeger.dev.storj.io/trace/54ea49bbbba7a774
FAIL (2025-12-08 18:33:32 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 2.006134248s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................]
duration:
701.728587ms
remote:
"tcp:35.212.10.183:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/3abf551f64a5a0f9
FAIL (2025-12-08 18:33:13 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 701.757492ms core satellite us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
475.831254ms
remote:
"tcp:35.207.121.91:443"
target:
https://eu1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/25680d6a319febe4
FAIL (2025-12-08 18:33:01 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 475.856343ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [X......................X........................X...........]
duration:
5.692252697s
trace:
https://jaeger.dev.storj.io/trace/323c8ad1ea235936
FAIL (2025-12-08 18:33:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.69228053s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................]
duration:
4.61105145s
trace:
https://jaeger.dev.storj.io/trace/2bd986c05c15b190
FAIL (2025-12-08 18:33:00 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
CDZRW6LEEY46BBQFTFZE5DYUXXXHUTIN3CVN63K65VQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.842730397 (74h24m40.713813432s) [noise=true , address=185.167.204.205:4032 , piece_id=NF4LIZE2CVGDLBP5O4TFDTJBINY76W2L6ARPSVV5AJXBREKTQS7Q, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.843290634 (74h24m40.714373669s) [noise=true , address=185.167.204.206:4064 , piece_id=2WY55HXXR5LPAR6XSD4GYYYA5BV3A2DPQ5LCPOL3UAWM4IU4ZIJA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.845386525 (74h24m40.716469571s) [noise=true , address=81.171.14.70:4029    , piece_id=3QQWIZOHYA4MRXNV37TOPRD3WWE3V45D2M474ADRLUHB47FLFUIQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.846892383 (74h24m40.717975428s) [noise=true , address=89.149.201.147:4010  , piece_id=XGAAAQTHKLOR3YLIQW3PKYQGKLES5ZHUHYMDHXFVQWELBQCLYFOQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.848772573 (74h24m40.719855618s) [noise=true , address=185.167.204.206:4049 , piece_id=7IYDBS2GFRUWYC5O62HTXPOL5IPXQ7NZPE4WHDBN5AEAEPECH2VA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.850191745 (74h24m40.72127478s) [noise=true , address=185.167.204.205:4017 , piece_id=2TUWBAIWW6E7GORGD6MBNQFIXUHTD2DTCOVBRJCJW4BFTOPLONCA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.851609155 (74h24m40.7226922s) [noise=true , address=89.149.201.147:4021  , piece_id=NX5FUM4724BSTFCEKRLLARMVFV7BFIXKT66XZ6RLIZTPMPWXN2AA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.852832574 (74h24m40.723915619s) [noise=true , address=81.171.14.70:4036    , piece_id=XBKXRAUGGAQVV62WSVGNPKXILMXBTIN2VWIAHJM37XLWLUSXWHNA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.854147148 (74h24m40.725230203s) [noise=true , address=81.171.14.70:4035    , piece_id=ULGXVDR7SYRD5FW5B52QXGXQRYUPU3FT4QFSCHWQDXRJ2JM5725Q, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:41.670064128 (74h24m41.541147173s) [upload=65114]: Next returned nil segment
Dec  8 17:53:41.670145664 (74h24m41.541228709s) [upload=65114]: Waiting for error group managing segments...
Dec  8 17:53:42.057773064 (74h24m41.928856099s) [noise=true , address=185.167.204.206:4065 , piece_id=7GQAOZEZ6OJKW3UK7U3NXYZM7VXGPHKW6UKHSTEUKJDNRZAYUA4A, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.058200909 (74h24m41.929283954s) [noise=true , address=185.167.204.206:4045 , piece_id=6O2I5IIAPPQFG66OPIOVY2GOHO4C5CM2UXMJHGY45WGQVZK3JGPQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.058523583 (74h24m41.929606628s) [noise=true , address=185.167.204.205:4034 , piece_id=6JYH3N55MJSMXE6VNGSZK7DZX7KUV7CPELNHIMPSFOXG4U2NXRTQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.060741817 (74h24m41.931824863s) [noise=true , address=185.167.204.205:4025 , piece_id=2MDDUA2MUJ6MXICI5H5O4KPAIRJUL7QLBIC4L6MIEKNDYQIO2EJQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.062816829 (74h24m41.933899864s) [noise=true , address=185.167.204.206:4072 , piece_id=X5XLPXKRFRIEQ32PZKY23AS4CFBXSC5NVCJF24IY2LQH4KLGG6RA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.064906518 (74h24m41.935989563s) [noise=true , address=185.167.204.206:4059 , piece_id=KPZWLMWMZA7GCT6JWIMSMLGUW6UVJ3FPWJKMSHP72C42TH5Z4ITA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.065603074 (74h24m41.93668611s) [noise=true , address=185.167.204.206:4047 , piece_id=4G5HDKWP26ER64KVLUXJ2IMQJUXMQ4YLNBYQAMY7MQMGE2PSREEA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.066790916 (74h24m41.937873951s) [noise=true , address=185.167.204.205:4017 , piece_id=2TUWBAIWW6E7GORGD6MBNQFIXUHTD2DTCOVBRJCJW4BFTOPLONCA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.075608666 (74h24m41.946691701s) [noise=true , address=185.167.204.206:4064 , piece_id=2WY55HXXR5LPAR6XSD4GYYYA5BV3A2DPQ5LCPOL3UAWM4IU4ZIJA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.076342644 (74h24m41.947425679s) [noise=true , address=185.167.204.205:4036 , piece_id=FKH55HUCJTNREM6PBSFGWURF5HIGWAM3U6OPQQVTRGEUA6K7JM2Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.077612843 (74h24m41.948695888s) [noise=true , address=185.167.204.205:4032 , piece_id=NF4LIZE2CVGDLBP5O4TFDTJBINY76W2L6ARPSVV5AJXBREKTQS7Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.078994745 (74h24m41.95007778s) [noise=true , address=185.167.204.206:4046 , piece_id=SDP7J6VHYTCJSKOHR4AXONUYML7VNYQIVATGTPENHWS4F4CO3A4A, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.082062506 (74h24m41.953145541s) [noise=true , address=185.167.204.206:4070 , piece_id=5ETK46YMRH46TMYDEYR4CPV3XDF3FS5MR5RSOYPZKD33EXMS5TQQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.096426879 (74h24m41.967509924s) [noise=true , address=185.167.204.206:4049 , piece_id=7IYDBS2GFRUWYC5O62HTXPOL5IPXQ7NZPE4WHDBN5AEAEPECH2VA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.766933610 (74h24m42.638016655s) [noise=true , address=81.171.14.70:4031    , piece_id=S4T3ORC533X2OW2Y3BQDAFSYXBTD3THLREQMMJ2G7QPM3UDGHRCQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.860649744 (74h24m42.731732789s) [noise=true , address=81.171.14.70:4029    , piece_id=3QQWIZOHYA4MRXNV37TOPRD3WWE3V45D2M474ADRLUHB47FLFUIQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.874184617 (74h24m42.745267652s) [noise=true , address=81.171.14.70:4035    , piece_id=ULGXVDR7SYRD5FW5B52QXGXQRYUPU3FT4QFSCHWQDXRJ2JM5725Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.254322126 (74h24m44.125405172s) [noise=true , address=89.149.201.147:4010  , piece_id=XGAAAQTHKLOR3YLIQW3PKYQGKLES5ZHUHYMDHXFVQWELBQCLYFOQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.671496219 (74h24m44.542579254s) [noise=true , address=81.171.14.70:4025    , piece_id=YX5EYUXYW4ZY3L6CH673H5BSHKUEKTUGGKVSFVENUVJBF3ONZUKA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.750460513 (74h24m44.621543558s) [noise=true , address=81.171.14.70:4033    , piece_id=LSIJ6NCTBALZH2QRPFFTLOXKKWF7J2E3NL2GIYXCDF2RC5KMNBPA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.792368875 (74h24m44.66345192s) [noise=true , address=81.171.14.70:4027    , piece_id=X53DOOCXSAHI64TN5GNSEOHBJK7GL6TLL6W7NJM5KQIVW3ITZ2BA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:45.851581290 (74h24m45.722664326s) [noise=true , address=81.171.14.70:4036    , piece_id=XBKXRAUGGAQVV62WSVGNPKXILMXBTIN2VWIAHJM37XLWLUSXWHNA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:46.587984219 (74h24m46.459067264s) [noise=true , address=89.149.201.147:4022  , piece_id=ZEX4GP43WCDZRW6LEEY46BBQFTFZE5DYUXXXHUTIN3CVN63K65VQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:46.808512099 (74h24m46.679595144s) [noise=true , address=81.171.14.70:4037    , piece_id=MHNTZCXG6COLI7J45SSYK6SZDVSSEBXOFFY7KEGP7NJUDEVLX7NA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:47.152614810 (74h24m47.023697845s) [noise=true , address=81.171.14.70:4038    , piece_id=VFW2FCTINXQNF3AFAFQW5AYWC2EZ2P52ERVUFU5DMH7672AWAQTA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:49.686376464 (74h24m49.557459499s) [noise=true , address=89.149.201.147:4021  , piece_id=NX5FUM4724BSTFCEKRLLARMVFV7BFIXKT66XZ6RLIZTPMPWXN2AA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:49.686397555 (74h24m49.55748059s) [seg_pos={0 0}, upload=65114]: Segment reached optimal threshold of 26 pieces.
Dec  8 17:53:49.686927403 (74h24m49.558010438s) [noise=true , address=89.149.201.147:4009  , piece_id=7SFNKC45B4BXNTU6IWIETH6P2JXVCCSO2ICFUBDFATHIU3NC4NOA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F): protocol: read tcp 188.166.225.170:60652->89.149.201.147:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec  8 17:53:49.687035098 (74h24m49.558118143s) [noise=true , address=89.149.201.147:4009  , piece_id=7SFNKC45B4BXNTU6IWIETH6P2JXVCCSO2ICFUBDFATHIU3NC4NOA, seg_pos={0 0}, upload=65114]: Long tail cancellation due to optimal threshold
Dec  8 17:53:49.687151320 (74h24m49.558234365s) [noise=true , address=89.149.201.147:4001  , piece_id=4HDC333BXRRPHPLSYQIKPP5UW2A26C776YDY2GFTQKSFI76SSLGA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: read tcp 188.166.225.170:58886->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec  8 17:53:49.687225261 (74h24m49.558308326s) [noise=true , address=89.149.201.147:4001  , piece_id=4HDC333BXRRPHPLSYQIKPP5UW2A26C776YDY2GFTQKSFI76SSLGA, seg_pos={0 0}, upload=65114]: Long tail cancellation due to optimal threshold
Dec  8 17:53:49.687301826 (74h24m49.558384872s) [seg_pos={0 0}, upload=65114]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec  8 17:53:49.687437044 (74h24m49.558520089s) [upload=65114]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec  8 17:53:49.687467202 (74h24m49.558550247s) [upload=65114]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec  8 18:01:19.776524628 (74h32m19.647607673s) [upload=65114]: Done uploading segments...
Dec  8 18:01:19.776620922 (74h32m19.647703967s) [upload=65114]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.611078992s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................]
duration:
703.836601ms
remote:
"tcp:35.212.10.183:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/3abf551f64a5a0f9
FAIL (2025-12-08 18:32:48 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 703.864725ms core satellite us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.............X........X.....................................]
duration:
5.173364898s
trace:
https://jaeger.dev.storj.io/trace/355abf85f96191de
FAIL (2025-12-08 18:32:41 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.173392971s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
3 out of 60 is failed (threshold=3) [..X...........................X.....................X.......]
duration:
6.098681147s
trace:
https://jaeger.dev.storj.io/trace/3d74ee1c8d24fd99
FAIL (2025-12-08 18:32:39 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 6.09872502s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................]
duration:
1.971238611s
trace:
https://jaeger.dev.storj.io/trace/54ea49bbbba7a774
FAIL (2025-12-08 18:32:30 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 1.971269078s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................]
duration:
699.732114ms
remote:
"tcp:35.212.10.183:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/3abf551f64a5a0f9
FAIL (2025-12-08 18:32:28 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 699.76178ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [.X......................X........................X..........]
duration:
5.950537294s
trace:
https://jaeger.dev.storj.io/trace/323c8ad1ea235936
FAIL (2025-12-08 18:32:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.950612787s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
6 out of 60 is failed (threshold=6) [.............X..X...............X............X...X......X...]
duration:
3.474692227s
trace:
https://jaeger.dev.storj.io/trace/1c7f30f7e7627ffd
FAIL (2025-12-08 18:32:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.474717935s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................]
duration:
5.26785867s
trace:
https://jaeger.dev.storj.io/trace/2bd986c05c15b190
FAIL (2025-12-08 18:32:23 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
CDZRW6LEEY46BBQFTFZE5DYUXXXHUTIN3CVN63K65VQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.842730397 (74h24m40.713813432s) [noise=true , address=185.167.204.205:4032 , piece_id=NF4LIZE2CVGDLBP5O4TFDTJBINY76W2L6ARPSVV5AJXBREKTQS7Q, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.843290634 (74h24m40.714373669s) [noise=true , address=185.167.204.206:4064 , piece_id=2WY55HXXR5LPAR6XSD4GYYYA5BV3A2DPQ5LCPOL3UAWM4IU4ZIJA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.845386525 (74h24m40.716469571s) [noise=true , address=81.171.14.70:4029    , piece_id=3QQWIZOHYA4MRXNV37TOPRD3WWE3V45D2M474ADRLUHB47FLFUIQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.846892383 (74h24m40.717975428s) [noise=true , address=89.149.201.147:4010  , piece_id=XGAAAQTHKLOR3YLIQW3PKYQGKLES5ZHUHYMDHXFVQWELBQCLYFOQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.848772573 (74h24m40.719855618s) [noise=true , address=185.167.204.206:4049 , piece_id=7IYDBS2GFRUWYC5O62HTXPOL5IPXQ7NZPE4WHDBN5AEAEPECH2VA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.850191745 (74h24m40.72127478s) [noise=true , address=185.167.204.205:4017 , piece_id=2TUWBAIWW6E7GORGD6MBNQFIXUHTD2DTCOVBRJCJW4BFTOPLONCA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.851609155 (74h24m40.7226922s) [noise=true , address=89.149.201.147:4021  , piece_id=NX5FUM4724BSTFCEKRLLARMVFV7BFIXKT66XZ6RLIZTPMPWXN2AA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.852832574 (74h24m40.723915619s) [noise=true , address=81.171.14.70:4036    , piece_id=XBKXRAUGGAQVV62WSVGNPKXILMXBTIN2VWIAHJM37XLWLUSXWHNA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.854147148 (74h24m40.725230203s) [noise=true , address=81.171.14.70:4035    , piece_id=ULGXVDR7SYRD5FW5B52QXGXQRYUPU3FT4QFSCHWQDXRJ2JM5725Q, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:41.670064128 (74h24m41.541147173s) [upload=65114]: Next returned nil segment
Dec  8 17:53:41.670145664 (74h24m41.541228709s) [upload=65114]: Waiting for error group managing segments...
Dec  8 17:53:42.057773064 (74h24m41.928856099s) [noise=true , address=185.167.204.206:4065 , piece_id=7GQAOZEZ6OJKW3UK7U3NXYZM7VXGPHKW6UKHSTEUKJDNRZAYUA4A, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.058200909 (74h24m41.929283954s) [noise=true , address=185.167.204.206:4045 , piece_id=6O2I5IIAPPQFG66OPIOVY2GOHO4C5CM2UXMJHGY45WGQVZK3JGPQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.058523583 (74h24m41.929606628s) [noise=true , address=185.167.204.205:4034 , piece_id=6JYH3N55MJSMXE6VNGSZK7DZX7KUV7CPELNHIMPSFOXG4U2NXRTQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.060741817 (74h24m41.931824863s) [noise=true , address=185.167.204.205:4025 , piece_id=2MDDUA2MUJ6MXICI5H5O4KPAIRJUL7QLBIC4L6MIEKNDYQIO2EJQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.062816829 (74h24m41.933899864s) [noise=true , address=185.167.204.206:4072 , piece_id=X5XLPXKRFRIEQ32PZKY23AS4CFBXSC5NVCJF24IY2LQH4KLGG6RA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.064906518 (74h24m41.935989563s) [noise=true , address=185.167.204.206:4059 , piece_id=KPZWLMWMZA7GCT6JWIMSMLGUW6UVJ3FPWJKMSHP72C42TH5Z4ITA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.065603074 (74h24m41.93668611s) [noise=true , address=185.167.204.206:4047 , piece_id=4G5HDKWP26ER64KVLUXJ2IMQJUXMQ4YLNBYQAMY7MQMGE2PSREEA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.066790916 (74h24m41.937873951s) [noise=true , address=185.167.204.205:4017 , piece_id=2TUWBAIWW6E7GORGD6MBNQFIXUHTD2DTCOVBRJCJW4BFTOPLONCA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.075608666 (74h24m41.946691701s) [noise=true , address=185.167.204.206:4064 , piece_id=2WY55HXXR5LPAR6XSD4GYYYA5BV3A2DPQ5LCPOL3UAWM4IU4ZIJA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.076342644 (74h24m41.947425679s) [noise=true , address=185.167.204.205:4036 , piece_id=FKH55HUCJTNREM6PBSFGWURF5HIGWAM3U6OPQQVTRGEUA6K7JM2Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.077612843 (74h24m41.948695888s) [noise=true , address=185.167.204.205:4032 , piece_id=NF4LIZE2CVGDLBP5O4TFDTJBINY76W2L6ARPSVV5AJXBREKTQS7Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.078994745 (74h24m41.95007778s) [noise=true , address=185.167.204.206:4046 , piece_id=SDP7J6VHYTCJSKOHR4AXONUYML7VNYQIVATGTPENHWS4F4CO3A4A, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.082062506 (74h24m41.953145541s) [noise=true , address=185.167.204.206:4070 , piece_id=5ETK46YMRH46TMYDEYR4CPV3XDF3FS5MR5RSOYPZKD33EXMS5TQQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.096426879 (74h24m41.967509924s) [noise=true , address=185.167.204.206:4049 , piece_id=7IYDBS2GFRUWYC5O62HTXPOL5IPXQ7NZPE4WHDBN5AEAEPECH2VA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.766933610 (74h24m42.638016655s) [noise=true , address=81.171.14.70:4031    , piece_id=S4T3ORC533X2OW2Y3BQDAFSYXBTD3THLREQMMJ2G7QPM3UDGHRCQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.860649744 (74h24m42.731732789s) [noise=true , address=81.171.14.70:4029    , piece_id=3QQWIZOHYA4MRXNV37TOPRD3WWE3V45D2M474ADRLUHB47FLFUIQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.874184617 (74h24m42.745267652s) [noise=true , address=81.171.14.70:4035    , piece_id=ULGXVDR7SYRD5FW5B52QXGXQRYUPU3FT4QFSCHWQDXRJ2JM5725Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.254322126 (74h24m44.125405172s) [noise=true , address=89.149.201.147:4010  , piece_id=XGAAAQTHKLOR3YLIQW3PKYQGKLES5ZHUHYMDHXFVQWELBQCLYFOQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.671496219 (74h24m44.542579254s) [noise=true , address=81.171.14.70:4025    , piece_id=YX5EYUXYW4ZY3L6CH673H5BSHKUEKTUGGKVSFVENUVJBF3ONZUKA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.750460513 (74h24m44.621543558s) [noise=true , address=81.171.14.70:4033    , piece_id=LSIJ6NCTBALZH2QRPFFTLOXKKWF7J2E3NL2GIYXCDF2RC5KMNBPA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.792368875 (74h24m44.66345192s) [noise=true , address=81.171.14.70:4027    , piece_id=X53DOOCXSAHI64TN5GNSEOHBJK7GL6TLL6W7NJM5KQIVW3ITZ2BA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:45.851581290 (74h24m45.722664326s) [noise=true , address=81.171.14.70:4036    , piece_id=XBKXRAUGGAQVV62WSVGNPKXILMXBTIN2VWIAHJM37XLWLUSXWHNA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:46.587984219 (74h24m46.459067264s) [noise=true , address=89.149.201.147:4022  , piece_id=ZEX4GP43WCDZRW6LEEY46BBQFTFZE5DYUXXXHUTIN3CVN63K65VQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:46.808512099 (74h24m46.679595144s) [noise=true , address=81.171.14.70:4037    , piece_id=MHNTZCXG6COLI7J45SSYK6SZDVSSEBXOFFY7KEGP7NJUDEVLX7NA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:47.152614810 (74h24m47.023697845s) [noise=true , address=81.171.14.70:4038    , piece_id=VFW2FCTINXQNF3AFAFQW5AYWC2EZ2P52ERVUFU5DMH7672AWAQTA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:49.686376464 (74h24m49.557459499s) [noise=true , address=89.149.201.147:4021  , piece_id=NX5FUM4724BSTFCEKRLLARMVFV7BFIXKT66XZ6RLIZTPMPWXN2AA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:49.686397555 (74h24m49.55748059s) [seg_pos={0 0}, upload=65114]: Segment reached optimal threshold of 26 pieces.
Dec  8 17:53:49.686927403 (74h24m49.558010438s) [noise=true , address=89.149.201.147:4009  , piece_id=7SFNKC45B4BXNTU6IWIETH6P2JXVCCSO2ICFUBDFATHIU3NC4NOA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F): protocol: read tcp 188.166.225.170:60652->89.149.201.147:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec  8 17:53:49.687035098 (74h24m49.558118143s) [noise=true , address=89.149.201.147:4009  , piece_id=7SFNKC45B4BXNTU6IWIETH6P2JXVCCSO2ICFUBDFATHIU3NC4NOA, seg_pos={0 0}, upload=65114]: Long tail cancellation due to optimal threshold
Dec  8 17:53:49.687151320 (74h24m49.558234365s) [noise=true , address=89.149.201.147:4001  , piece_id=4HDC333BXRRPHPLSYQIKPP5UW2A26C776YDY2GFTQKSFI76SSLGA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: read tcp 188.166.225.170:58886->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec  8 17:53:49.687225261 (74h24m49.558308326s) [noise=true , address=89.149.201.147:4001  , piece_id=4HDC333BXRRPHPLSYQIKPP5UW2A26C776YDY2GFTQKSFI76SSLGA, seg_pos={0 0}, upload=65114]: Long tail cancellation due to optimal threshold
Dec  8 17:53:49.687301826 (74h24m49.558384872s) [seg_pos={0 0}, upload=65114]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec  8 17:53:49.687437044 (74h24m49.558520089s) [upload=65114]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec  8 17:53:49.687467202 (74h24m49.558550247s) [upload=65114]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec  8 18:01:19.776524628 (74h32m19.647607673s) [upload=65114]: Done uploading segments...
Dec  8 18:01:19.776620922 (74h32m19.647703967s) [upload=65114]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.267889721s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..............X........X....................................]
duration:
5.157293523s
trace:
https://jaeger.dev.storj.io/trace/355abf85f96191de
FAIL (2025-12-08 18:32:21 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.157355393s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................]
duration:
1.989314719s
trace:
https://jaeger.dev.storj.io/trace/54ea49bbbba7a774
FAIL (2025-12-08 18:32:14 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 1.989431712s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................]
duration:
704.949841ms
remote:
"tcp:35.212.10.183:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/3abf551f64a5a0f9
FAIL (2025-12-08 18:32:10 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 704.976119ms core satellite us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
482.48255ms
remote:
"tcp:35.207.121.91:443"
target:
https://eu1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/25680d6a319febe4
FAIL (2025-12-08 18:32:10 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 482.506846ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................]
duration:
1.990124682s
trace:
https://jaeger.dev.storj.io/trace/54ea49bbbba7a774
FAIL (2025-12-08 18:31:44 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 1.990154178s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
3 out of 60 is failed (threshold=3) [...X...........................X.....................X......]
duration:
5.871018055s
trace:
https://jaeger.dev.storj.io/trace/3d74ee1c8d24fd99
FAIL (2025-12-08 18:31:41 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.871045807s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................]
duration:
26.089436784s
trace:
https://jaeger.dev.storj.io/trace/2bd986c05c15b190
FAIL (2025-12-08 18:31:32 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
CDZRW6LEEY46BBQFTFZE5DYUXXXHUTIN3CVN63K65VQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.842730397 (74h24m40.713813432s) [noise=true , address=185.167.204.205:4032 , piece_id=NF4LIZE2CVGDLBP5O4TFDTJBINY76W2L6ARPSVV5AJXBREKTQS7Q, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.843290634 (74h24m40.714373669s) [noise=true , address=185.167.204.206:4064 , piece_id=2WY55HXXR5LPAR6XSD4GYYYA5BV3A2DPQ5LCPOL3UAWM4IU4ZIJA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.845386525 (74h24m40.716469571s) [noise=true , address=81.171.14.70:4029    , piece_id=3QQWIZOHYA4MRXNV37TOPRD3WWE3V45D2M474ADRLUHB47FLFUIQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.846892383 (74h24m40.717975428s) [noise=true , address=89.149.201.147:4010  , piece_id=XGAAAQTHKLOR3YLIQW3PKYQGKLES5ZHUHYMDHXFVQWELBQCLYFOQ, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.848772573 (74h24m40.719855618s) [noise=true , address=185.167.204.206:4049 , piece_id=7IYDBS2GFRUWYC5O62HTXPOL5IPXQ7NZPE4WHDBN5AEAEPECH2VA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.850191745 (74h24m40.72127478s) [noise=true , address=185.167.204.205:4017 , piece_id=2TUWBAIWW6E7GORGD6MBNQFIXUHTD2DTCOVBRJCJW4BFTOPLONCA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.851609155 (74h24m40.7226922s) [noise=true , address=89.149.201.147:4021  , piece_id=NX5FUM4724BSTFCEKRLLARMVFV7BFIXKT66XZ6RLIZTPMPWXN2AA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.852832574 (74h24m40.723915619s) [noise=true , address=81.171.14.70:4036    , piece_id=XBKXRAUGGAQVV62WSVGNPKXILMXBTIN2VWIAHJM37XLWLUSXWHNA, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:40.854147148 (74h24m40.725230203s) [noise=true , address=81.171.14.70:4035    , piece_id=ULGXVDR7SYRD5FW5B52QXGXQRYUPU3FT4QFSCHWQDXRJ2JM5725Q, seg_pos={0 0}, upload=65114]: Uploading piece...
Dec  8 17:53:41.670064128 (74h24m41.541147173s) [upload=65114]: Next returned nil segment
Dec  8 17:53:41.670145664 (74h24m41.541228709s) [upload=65114]: Waiting for error group managing segments...
Dec  8 17:53:42.057773064 (74h24m41.928856099s) [noise=true , address=185.167.204.206:4065 , piece_id=7GQAOZEZ6OJKW3UK7U3NXYZM7VXGPHKW6UKHSTEUKJDNRZAYUA4A, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.058200909 (74h24m41.929283954s) [noise=true , address=185.167.204.206:4045 , piece_id=6O2I5IIAPPQFG66OPIOVY2GOHO4C5CM2UXMJHGY45WGQVZK3JGPQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.058523583 (74h24m41.929606628s) [noise=true , address=185.167.204.205:4034 , piece_id=6JYH3N55MJSMXE6VNGSZK7DZX7KUV7CPELNHIMPSFOXG4U2NXRTQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.060741817 (74h24m41.931824863s) [noise=true , address=185.167.204.205:4025 , piece_id=2MDDUA2MUJ6MXICI5H5O4KPAIRJUL7QLBIC4L6MIEKNDYQIO2EJQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.062816829 (74h24m41.933899864s) [noise=true , address=185.167.204.206:4072 , piece_id=X5XLPXKRFRIEQ32PZKY23AS4CFBXSC5NVCJF24IY2LQH4KLGG6RA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.064906518 (74h24m41.935989563s) [noise=true , address=185.167.204.206:4059 , piece_id=KPZWLMWMZA7GCT6JWIMSMLGUW6UVJ3FPWJKMSHP72C42TH5Z4ITA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.065603074 (74h24m41.93668611s) [noise=true , address=185.167.204.206:4047 , piece_id=4G5HDKWP26ER64KVLUXJ2IMQJUXMQ4YLNBYQAMY7MQMGE2PSREEA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.066790916 (74h24m41.937873951s) [noise=true , address=185.167.204.205:4017 , piece_id=2TUWBAIWW6E7GORGD6MBNQFIXUHTD2DTCOVBRJCJW4BFTOPLONCA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.075608666 (74h24m41.946691701s) [noise=true , address=185.167.204.206:4064 , piece_id=2WY55HXXR5LPAR6XSD4GYYYA5BV3A2DPQ5LCPOL3UAWM4IU4ZIJA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.076342644 (74h24m41.947425679s) [noise=true , address=185.167.204.205:4036 , piece_id=FKH55HUCJTNREM6PBSFGWURF5HIGWAM3U6OPQQVTRGEUA6K7JM2Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.077612843 (74h24m41.948695888s) [noise=true , address=185.167.204.205:4032 , piece_id=NF4LIZE2CVGDLBP5O4TFDTJBINY76W2L6ARPSVV5AJXBREKTQS7Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.078994745 (74h24m41.95007778s) [noise=true , address=185.167.204.206:4046 , piece_id=SDP7J6VHYTCJSKOHR4AXONUYML7VNYQIVATGTPENHWS4F4CO3A4A, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.082062506 (74h24m41.953145541s) [noise=true , address=185.167.204.206:4070 , piece_id=5ETK46YMRH46TMYDEYR4CPV3XDF3FS5MR5RSOYPZKD33EXMS5TQQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.096426879 (74h24m41.967509924s) [noise=true , address=185.167.204.206:4049 , piece_id=7IYDBS2GFRUWYC5O62HTXPOL5IPXQ7NZPE4WHDBN5AEAEPECH2VA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.766933610 (74h24m42.638016655s) [noise=true , address=81.171.14.70:4031    , piece_id=S4T3ORC533X2OW2Y3BQDAFSYXBTD3THLREQMMJ2G7QPM3UDGHRCQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.860649744 (74h24m42.731732789s) [noise=true , address=81.171.14.70:4029    , piece_id=3QQWIZOHYA4MRXNV37TOPRD3WWE3V45D2M474ADRLUHB47FLFUIQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:42.874184617 (74h24m42.745267652s) [noise=true , address=81.171.14.70:4035    , piece_id=ULGXVDR7SYRD5FW5B52QXGXQRYUPU3FT4QFSCHWQDXRJ2JM5725Q, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.254322126 (74h24m44.125405172s) [noise=true , address=89.149.201.147:4010  , piece_id=XGAAAQTHKLOR3YLIQW3PKYQGKLES5ZHUHYMDHXFVQWELBQCLYFOQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.671496219 (74h24m44.542579254s) [noise=true , address=81.171.14.70:4025    , piece_id=YX5EYUXYW4ZY3L6CH673H5BSHKUEKTUGGKVSFVENUVJBF3ONZUKA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.750460513 (74h24m44.621543558s) [noise=true , address=81.171.14.70:4033    , piece_id=LSIJ6NCTBALZH2QRPFFTLOXKKWF7J2E3NL2GIYXCDF2RC5KMNBPA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:44.792368875 (74h24m44.66345192s) [noise=true , address=81.171.14.70:4027    , piece_id=X53DOOCXSAHI64TN5GNSEOHBJK7GL6TLL6W7NJM5KQIVW3ITZ2BA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:45.851581290 (74h24m45.722664326s) [noise=true , address=81.171.14.70:4036    , piece_id=XBKXRAUGGAQVV62WSVGNPKXILMXBTIN2VWIAHJM37XLWLUSXWHNA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:46.587984219 (74h24m46.459067264s) [noise=true , address=89.149.201.147:4022  , piece_id=ZEX4GP43WCDZRW6LEEY46BBQFTFZE5DYUXXXHUTIN3CVN63K65VQ, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:46.808512099 (74h24m46.679595144s) [noise=true , address=81.171.14.70:4037    , piece_id=MHNTZCXG6COLI7J45SSYK6SZDVSSEBXOFFY7KEGP7NJUDEVLX7NA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:47.152614810 (74h24m47.023697845s) [noise=true , address=81.171.14.70:4038    , piece_id=VFW2FCTINXQNF3AFAFQW5AYWC2EZ2P52ERVUFU5DMH7672AWAQTA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:49.686376464 (74h24m49.557459499s) [noise=true , address=89.149.201.147:4021  , piece_id=NX5FUM4724BSTFCEKRLLARMVFV7BFIXKT66XZ6RLIZTPMPWXN2AA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: <nil>
Dec  8 17:53:49.686397555 (74h24m49.55748059s) [seg_pos={0 0}, upload=65114]: Segment reached optimal threshold of 26 pieces.
Dec  8 17:53:49.686927403 (74h24m49.558010438s) [noise=true , address=89.149.201.147:4009  , piece_id=7SFNKC45B4BXNTU6IWIETH6P2JXVCCSO2ICFUBDFATHIU3NC4NOA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F): protocol: read tcp 188.166.225.170:60652->89.149.201.147:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec  8 17:53:49.687035098 (74h24m49.558118143s) [noise=true , address=89.149.201.147:4009  , piece_id=7SFNKC45B4BXNTU6IWIETH6P2JXVCCSO2ICFUBDFATHIU3NC4NOA, seg_pos={0 0}, upload=65114]: Long tail cancellation due to optimal threshold
Dec  8 17:53:49.687151320 (74h24m49.558234365s) [noise=true , address=89.149.201.147:4001  , piece_id=4HDC333BXRRPHPLSYQIKPP5UW2A26C776YDY2GFTQKSFI76SSLGA, seg_pos={0 0}, upload=65114]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: read tcp 188.166.225.170:58886->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec  8 17:53:49.687225261 (74h24m49.558308326s) [noise=true , address=89.149.201.147:4001  , piece_id=4HDC333BXRRPHPLSYQIKPP5UW2A26C776YDY2GFTQKSFI76SSLGA, seg_pos={0 0}, upload=65114]: Long tail cancellation due to optimal threshold
Dec  8 17:53:49.687301826 (74h24m49.558384872s) [seg_pos={0 0}, upload=65114]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec  8 17:53:49.687437044 (74h24m49.558520089s) [upload=65114]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec  8 17:53:49.687467202 (74h24m49.558550247s) [upload=65114]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec  8 18:01:19.776524628 (74h32m19.647607673s) [upload=65114]: Done uploading segments...
Dec  8 18:01:19.776620922 (74h32m19.647703967s) [upload=65114]: Upload finished. err: metaclient: context deadline exceeded
Duration: 26.089467883s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...............X........X...................................]
duration:
5.682701553s
trace:
https://jaeger.dev.storj.io/trace/355abf85f96191de
FAIL (2025-12-08 18:31:27 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.682743232s core uplink eu1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................]
duration:
694.883563ms
remote:
"tcp:35.212.10.183:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/3abf551f64a5a0f9
FAIL (2025-12-08 18:31:25 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 694.911106ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
6 out of 60 is failed (threshold=6) [..............X..X...............X............X...X......X..]
duration:
3.52684307s
trace:
https://jaeger.dev.storj.io/trace/1c7f30f7e7627ffd
FAIL (2025-12-08 18:31:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.526871726s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [..X......................X........................X.........]
duration:
5.675155424s
trace:
https://jaeger.dev.storj.io/trace/323c8ad1ea235936
FAIL (2025-12-08 18:31:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.675188476s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................]
duration:
2.01060414s
trace:
https://jaeger.dev.storj.io/trace/54ea49bbbba7a774
FAIL (2025-12-08 18:31:15 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 2.010633985s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
3 out of 60 is failed (threshold=3) [....X...........................X.....................X.....]
duration:
10.753671521s
trace:
https://jaeger.dev.storj.io/trace/3d74ee1c8d24fd99
FAIL (2025-12-08 18:31:14 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 10.753702361s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
483.135083ms
remote:
"tcp:35.207.121.91:443"
target:
https://eu1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/25680d6a319febe4
FAIL (2025-12-08 18:31:11 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 483.160351ms core satellite eu1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................]
duration:
699.560789ms
remote:
"tcp:35.212.10.183:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/3abf551f64a5a0f9
FAIL (2025-12-08 18:31:04 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 699.594895ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................]
duration:
2.003537355s
trace:
https://jaeger.dev.storj.io/trace/54ea49bbbba7a774
FAIL (2025-12-08 18:30:59 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 2.003562022s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [................X........X..................................]
duration:
5.13453428s
trace:
https://jaeger.dev.storj.io/trace/355abf85f96191de
FAIL (2025-12-08 18:30:57 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.134553626s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
3 out of 60 is failed (threshold=3) [.....X...........................X.....................X....]
duration:
5.972129938s
trace:
https://jaeger.dev.storj.io/trace/3d74ee1c8d24fd99
FAIL (2025-12-08 18:30:53 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.972180435s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
6 out of 60 is failed (threshold=6) [...............X..X...............X............X...X......X.]
duration:
4.182647889s
trace:
https://jaeger.dev.storj.io/trace/1c7f30f7e7627ffd
FAIL (2025-12-08 18:30:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.182676756s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [...X......................X........................X........]
duration:
6.083412964s
trace:
https://jaeger.dev.storj.io/trace/323c8ad1ea235936
FAIL (2025-12-08 18:30:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.083427814s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................]
duration:
5.973194667s
remote:
"tcp:35.212.10.183:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/3abf551f64a5a0f9
FAIL (2025-12-08 18:30:03 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 5.973237017s core satellite us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
474.913678ms
remote:
"tcp:35.207.121.91:443"
target:
https://eu1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/25680d6a319febe4
FAIL (2025-12-08 18:29:59 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 474.94121ms core satellite eu1