Orbiter History

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 632.27115ms remote: "tcp:34.120.119.150:443" target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/5e0e211d0669c100 FAIL (2025-11-09 01:01:17 UTC) orbiter warning: Get "https://version.storj.io/": tls: failed to verify certificate: x509: certificate is valid for objectmount.com, www.objectmount.com, objectmount.ai, www.objectmount.ai, objectmount.io, www.objectmount.io, objectmount.net, www.objectmount.net, objectmount.org, www.objectmount.org, not version.storj.io Duration: 632.29231ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......X............................................X.......] duration: 3.51170942s trace: https://jaeger.dev.storj.io/trace/7981fc52a0541ef5 FAIL (2025-11-09 01:01:09 UTC) orbiter warning: context deadline exceeded Duration: 3.511747312s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........X............................................X......] duration: 3.455759138s trace: https://jaeger.dev.storj.io/trace/7981fc52a0541ef5 FAIL (2025-11-09 01:00:47 UTC) orbiter warning: context deadline exceeded Duration: 3.455790977s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.918206861s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 01:00:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.91823874s core uplink us-select-1
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 724.140286ms remote: "tcp:34.120.119.150:443" target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/5e0e211d0669c100 FAIL (2025-11-09 01:00:21 UTC) orbiter warning: Get "https://version.storj.io/": tls: failed to verify certificate: x509: certificate is valid for objectmount.com, www.objectmount.com, objectmount.ai, www.objectmount.ai, objectmount.io, www.objectmount.io, objectmount.net, www.objectmount.net, objectmount.org, www.objectmount.org, not version.storj.io Duration: 724.167148ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [X........X............................................X.....] duration: 3.389647711s trace: https://jaeger.dev.storj.io/trace/7981fc52a0541ef5 FAIL (2025-11-09 00:59:48 UTC) orbiter warning: context deadline exceeded Duration: 3.389673901s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.854006102s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:59:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.854031511s core uplink us-select-1
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 726.198907ms remote: "tcp:34.120.119.150:443" target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/5e0e211d0669c100 FAIL (2025-11-09 00:59:31 UTC) orbiter warning: Get "https://version.storj.io/": tls: failed to verify certificate: x509: certificate is valid for objectmount.com, www.objectmount.com, objectmount.ai, www.objectmount.ai, objectmount.io, www.objectmount.io, objectmount.net, www.objectmount.net, objectmount.org, www.objectmount.org, not version.storj.io Duration: 726.228073ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.X........X............................................X....] duration: 3.428867483s trace: https://jaeger.dev.storj.io/trace/7981fc52a0541ef5 FAIL (2025-11-09 00:58:55 UTC) orbiter warning: context deadline exceeded Duration: 3.428886018s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.942544805s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:58:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.942571917s core uplink us-select-1
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 632.986431ms remote: "tcp:34.120.119.150:443" target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/5e0e211d0669c100 FAIL (2025-11-09 00:58:35 UTC) orbiter warning: Get "https://version.storj.io/": tls: failed to verify certificate: x509: certificate is valid for objectmount.com, www.objectmount.com, objectmount.ai, www.objectmount.ai, objectmount.io, www.objectmount.io, objectmount.net, www.objectmount.net, objectmount.org, www.objectmount.org, not version.storj.io Duration: 633.013113ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..X........X............................................X...] duration: 3.619439678s trace: https://jaeger.dev.storj.io/trace/7981fc52a0541ef5 FAIL (2025-11-09 00:58:22 UTC) orbiter warning: context deadline exceeded Duration: 3.619457041s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.325360347s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:57:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 6.325384522s core uplink us-select-1
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 622.364115ms remote: "tcp:34.120.119.150:443" target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/5e0e211d0669c100 FAIL (2025-11-09 00:57:31 UTC) orbiter warning: Get "https://version.storj.io/": tls: failed to verify certificate: x509: certificate is valid for objectmount.com, www.objectmount.com, objectmount.ai, www.objectmount.ai, objectmount.io, www.objectmount.io, objectmount.net, www.objectmount.net, objectmount.org, www.objectmount.org, not version.storj.io Duration: 622.39804ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...X........X............................................X..] duration: 3.509862481s trace: https://jaeger.dev.storj.io/trace/7981fc52a0541ef5 FAIL (2025-11-09 00:57:31 UTC) orbiter warning: context deadline exceeded Duration: 3.510406138s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [....X........X............................................X.] duration: 3.362706591s trace: https://jaeger.dev.storj.io/trace/7981fc52a0541ef5 FAIL (2025-11-09 00:56:44 UTC) orbiter warning: context deadline exceeded Duration: 3.362756475s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.969754586s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:56:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.969775806s core uplink us-select-1
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 632.001157ms remote: "tcp:34.120.119.150:443" target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/5e0e211d0669c100 FAIL (2025-11-09 00:56:23 UTC) orbiter warning: Get "https://version.storj.io/": tls: failed to verify certificate: x509: certificate is valid for objectmount.com, www.objectmount.com, objectmount.ai, www.objectmount.ai, objectmount.io, www.objectmount.io, objectmount.net, www.objectmount.net, objectmount.org, www.objectmount.org, not version.storj.io Duration: 632.03428ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.....X........X............................................X] duration: 5.000917855s trace: https://jaeger.dev.storj.io/trace/7981fc52a0541ef5 FAIL (2025-11-09 00:56:05 UTC) orbiter warning: context deadline exceeded Duration: 5.000931321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.969118716s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:55:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.969140409s core uplink us-select-1
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 618.964762ms remote: "tcp:34.120.119.150:443" target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/5e0e211d0669c100 FAIL (2025-11-09 00:55:28 UTC) orbiter warning: Get "https://version.storj.io/": tls: failed to verify certificate: x509: certificate is valid for objectmount.com, www.objectmount.com, objectmount.ai, www.objectmount.ai, objectmount.io, www.objectmount.io, objectmount.net, www.objectmount.net, objectmount.org, www.objectmount.org, not version.storj.io Duration: 619.004669ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......X........X............................................] duration: 3.561387743s trace: https://jaeger.dev.storj.io/trace/538482b67f052294 FAIL (2025-11-09 00:55:05 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.561489197s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.07719844s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:55:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 6.077221282s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 317.879441ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4f1ed0c21ac068b9 FAIL (2025-11-09 00:54:50 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:46250->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 317.901743ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......X........X...........................................] duration: 3.489749223s trace: https://jaeger.dev.storj.io/trace/538482b67f052294 FAIL (2025-11-09 00:54:33 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.489778569s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 629.549809ms remote: "tcp:34.120.119.150:443" target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/5e0e211d0669c100 FAIL (2025-11-09 00:54:22 UTC) orbiter warning: Get "https://version.storj.io/": tls: failed to verify certificate: x509: certificate is valid for objectmount.com, www.objectmount.com, objectmount.ai, www.objectmount.ai, objectmount.io, www.objectmount.io, objectmount.net, www.objectmount.net, objectmount.org, www.objectmount.org, not version.storj.io Duration: 629.581108ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........X........X..........................................] duration: 3.42185571s trace: https://jaeger.dev.storj.io/trace/538482b67f052294 FAIL (2025-11-09 00:54:12 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.421875928s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........X........X.........................................] duration: 3.455390761s trace: https://jaeger.dev.storj.io/trace/538482b67f052294 FAIL (2025-11-09 00:53:56 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.455418543s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.979746254s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:53:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.979778956s core uplink us-select-1
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 632.350151ms remote: "tcp:34.120.119.150:443" target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/5e0e211d0669c100 FAIL (2025-11-09 00:53:45 UTC) orbiter warning: Get "https://version.storj.io/": tls: failed to verify certificate: x509: certificate is valid for objectmount.com, www.objectmount.com, objectmount.ai, www.objectmount.ai, objectmount.io, www.objectmount.io, objectmount.net, www.objectmount.net, objectmount.org, www.objectmount.org, not version.storj.io Duration: 632.383956ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 298.361685ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4f1ed0c21ac068b9 FAIL (2025-11-09 00:53:44 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:46250->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 298.419316ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 372.612381ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4f1ed0c21ac068b9 FAIL (2025-11-09 00:53:20 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:46250->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 372.650003ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.082295596s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:53:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 6.082325051s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........X........X........................................] duration: 3.468539505s trace: https://jaeger.dev.storj.io/trace/538482b67f052294 FAIL (2025-11-09 00:53:03 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.468570123s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 325.753204ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4f1ed0c21ac068b9 FAIL (2025-11-09 00:52:35 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:46250->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 325.770267ms core website
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 9.476334ms remote: "tcp:34.120.119.150:443" target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/5e0e211d0669c100 FAIL (2025-11-09 00:52:31 UTC) orbiter warning: Get "https://version.storj.io/": tls: failed to verify certificate: x509: certificate is valid for objectmount.com, www.objectmount.com, objectmount.ai, www.objectmount.ai, objectmount.io, www.objectmount.io, objectmount.net, www.objectmount.net, objectmount.org, www.objectmount.org, not version.storj.io Duration: 9.499619ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.951226013s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:52:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.951257562s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 307.354297ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4f1ed0c21ac068b9 FAIL (2025-11-09 00:51:56 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:46250->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 307.383793ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........X........X.......................................] duration: 3.451756905s trace: https://jaeger.dev.storj.io/trace/538482b67f052294 FAIL (2025-11-09 00:51:53 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.451789689s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.842356635s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:51:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.84238509s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............X........X......................................] duration: 3.447612244s trace: https://jaeger.dev.storj.io/trace/538482b67f052294 FAIL (2025-11-09 00:50:55 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.447641581s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 322.208483ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4f1ed0c21ac068b9 FAIL (2025-11-09 00:50:52 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:46250->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 322.240113ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............X........X.....................................] duration: 3.499187973s trace: https://jaeger.dev.storj.io/trace/538482b67f052294 FAIL (2025-11-09 00:50:19 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.499210187s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.892916201s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:49:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.892949074s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 307.553018ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4f1ed0c21ac068b9 FAIL (2025-11-09 00:49:47 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:46250->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 307.581942ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.916236868s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:49:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.916261887s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 379.931789ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4f1ed0c21ac068b9 FAIL (2025-11-09 00:49:26 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:46250->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 379.95846ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............X........X....................................] duration: 3.447260817s trace: https://jaeger.dev.storj.io/trace/538482b67f052294 FAIL (2025-11-09 00:49:08 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.447290954s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.997923552s trace: https://jaeger.dev.storj.io/trace/15a102436a900d91 FAIL (2025-11-09 00:48:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== pload=201217]: Uploading piece... Nov 9 00:15:07.882409862 (228h34m39.578258615s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.882737906 (228h34m39.578586659s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883235645 (228h34m39.579084398s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883270351 (228h34m39.579119104s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883583637 (228h34m39.57943239s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:07.883903466 (228h34m39.579752219s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Uploading piece... Nov 9 00:15:09.669212422 (228h34m41.365061175s) [upload=201217]: Next returned nil segment Nov 9 00:15:09.669230136 (228h34m41.365078889s) [upload=201217]: Waiting for error group managing segments... Nov 9 00:15:09.979393119 (228h34m41.675241862s) [noise=true , address=45.91.116.10:4082 , piece_id=3VLUPZU4BJ6Q7BR5CGDFTGVHWR5UTYOHC35ZWILT3OTL4V2ETZGQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:09.982783635 (228h34m41.678632388s) [noise=true , address=45.91.116.9:4263 , piece_id=Z7MGGEQ63C5QOKI7BAAQCCJPJPLBM2MG2MBROD7GL4RRNCFVJJOQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.004031905 (228h34m41.699880668s) [noise=true , address=45.91.116.6:4973 , piece_id=2R4VUB2FG7IIY3KSBYSZFU7VTBGNS662DQ2AFOH7KDHNYSBGNUBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:10.072857382 (228h34m41.768706145s) [noise=true , address=77.90.31.10:4520 , piece_id=QFF5PLOA5UXWFJCDC3FYFVN4YJQX3GGVVUH3ACCNJ4LB5FT3BPAQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: <nil> Nov 9 00:15:36.939056286 (228h35m8.634905039s) [noise=true , address=145.79.227.28:4726 , piece_id=EGLGRHWIZETHAYGPUIMSPEGOB3HXC2YMC32EGJIXSW7TQ4KGM2GA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12ACUxMRYJG7bzfXw6i1ioF6FSzbQvuWmZgrgcH8XhCxcncycL4, address:145.79.227.28:4726): protocol: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43478->145.79.227.28:4726: use of closed network connection Nov 9 00:15:36.939107635 (228h35m8.634956398s) [noise=true , address=145.79.227.27:4690 , piece_id=ZQ2KUEEUTKDO44XPN5YBKH6BUTLWA7TSCZRJ3QLLGTT4DWYHGP6Q, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44312->145.79.227.27:4690: use of closed network connection Nov 9 00:15:36.939340538 (228h35m8.635189292s) [noise=true , address=145.79.227.10:4401 , piece_id=FKLG63DNHBF43ZT3MDH2D6GEDURMW6XA335EZND2LSQITYI5TYHQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40318->145.79.227.10:4401: use of closed network connection Nov 9 00:15:36.939371116 (228h35m8.635219879s) [noise=true , address=145.79.227.15:4876 , piece_id=5XA3I6VEOLUYSQK374SW52H5LNMHEZX2HVOC6L6FFMAKGJV4RXTQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54874->145.79.227.15:4876: use of closed network connection Nov 9 00:15:36.939473351 (228h35m8.635322104s) [noise=true , address=145.79.227.21:4473 , piece_id=3UNC3F6P7ERWHHZQTVSCIP4SBO6XL72I5F4ID5SOCIKTRFEWM6XQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40864->145.79.227.21:4473: use of closed network connection Nov 9 00:15:36.939551320 (228h35m8.635400073s) [noise=true , address=145.79.227.22:4509 , piece_id=QWLYFFMJSIQLRC5OWIOTWLIYUOZY34VJDQTMQJOXTCQ4DWYFCBVA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: expected piece hash; context deadline exceeded; EOF Nov 9 00:15:36.939605372 (228h35m8.635454125s) [noise=true , address=145.79.227.8:4330 , piece_id=M7O6MM4DRNV7Y63AGYACAKZ6674HNG777XVUGEAZ7XVEDIBBYTVQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW, address:145.79.227.8:4330): protocol: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->145.79.227.8:4330: use of closed network connection Nov 9 00:15:36.939692238 (228h35m8.635540991s) [noise=true , address=145.79.227.8:4331 , piece_id=RDWGYWFEQCZIPKTSAG235PUJJDIP4BCDNJTX237RLFPTE2C45IBA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36256->145.79.227.8:4331: use of closed network connection Nov 9 00:15:36.939816855 (228h35m8.635665608s) [noise=true , address=145.79.227.5:4222 , piece_id=F26B25SIZ7TBDEKYDU34P477KIHHJUIHSYKRHGHOKKJVSQNNC3FA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44504->145.79.227.5:4222: use of closed network connection Nov 9 00:15:36.939964185 (228h35m8.635812938s) [noise=true , address=145.79.227.9:4365 , piece_id=GYEVA6LDJIQJJ6AVSTAXAVD2JDQ6CBQERQJJMBTW5OV4EZMVTOKA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, address:145.79.227.9:4365): protocol: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45216->145.79.227.9:4365: use of closed network connection Nov 9 00:15:36.940093301 (228h35m8.635942054s) [noise=true , address=145.79.227.13:4804 , piece_id=MTPRWDDQQ5LMOTNCPOGKDH5IA57BICDAWSNOQ3H6BEBSZJSDOICQ, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40310->145.79.227.13:4804: use of closed network connection Nov 9 00:15:36.940217358 (228h35m8.636066111s) [noise=true , address=145.79.227.25:4618 , piece_id=J7WJIE3QGIMDMAPSGEDMOUN23GX356DCB3LGFN7XJUNYK6CCLTMA, seg_pos={0 0}, upload=201217]: Done uploading piece. err: ecclient: upload failed (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8, address:145.79.227.25:4618): protocol: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52014->145.79.227.25:4618: use of closed network connection Nov 9 00:15:36.940262043 (228h35m8.636110786s) [seg_pos={0 0}, upload=201217]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 9 00:15:37.181912046 (228h35m8.877760799s) [upload=201217]: Done uploading segments... Nov 9 00:15:37.181957073 (228h35m8.877805816s) [upload=201217]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 5.998000148s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 300.433059ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4f1ed0c21ac068b9 FAIL (2025-11-09 00:48:28 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:46250->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 300.531647ms core website

Orbiter Check History

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)