Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.817983701s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:42:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.818007206s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [XX.XX........................................X..............] duration: 3.392687365s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:42:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.392747822s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.715730505s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:41:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.715758878s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [XXX.XX........................................X.............] duration: 3.391568183s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:41:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.391593311s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.766196667s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:41:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.766228699s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [.XXX.XX........................................X............] duration: 3.556237561s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:40:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.556264193s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.831614531s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:39:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.831641853s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [..XXX.XX........................................X...........] duration: 3.674673344s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:39:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.674696838s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [...XXX.XX........................................X..........] duration: 3.6565259s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:38:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.656542752s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.786294663s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:38:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.786320504s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [....XXX.XX........................................X.........] duration: 3.619200219s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:38:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.619227199s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [X....XXX.XX........................................X........] duration: 3.740488509s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:37:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.740507675s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.827755654s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:37:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.827790791s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [.X....XXX.XX........................................X.......] duration: 3.67891777s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:37:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.678963389s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.753125445s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:36:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.753151496s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.706981234s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:36:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.707011742s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [..X....XXX.XX........................................X......] duration: 3.735505707s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:36:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.735540354s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.778687042s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:35:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.778703003s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.772034231s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:35:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.772051715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.754764469s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:35:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.754794568s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [...X....XXX.XX........................................X.....] duration: 3.420813658s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:35:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.420848836s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [....X....XXX.XX........................................X....] duration: 3.479416596s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:34:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.479441884s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.682105205s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:34:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.682132998s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.660114532s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:34:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.660143808s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [X....X....XXX.XX........................................X...] duration: 3.520683649s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:33:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.520757812s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.753013864s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:33:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.753041536s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [XX....X....XXX.XX........................................X..] duration: 3.439122361s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:32:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.439150354s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.739516089s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:32:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.739543142s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.858695541s trace: https://jaeger.dev.storj.io/trace/7b008eb7ca241c4a FAIL (2026-04-05 05:32:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.858727161s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [.XX....X....XXX.XX........................................X.] duration: 3.435981291s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:31:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.436003422s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 51.921405565s trace: https://jaeger.dev.storj.io/trace/7b008eb7ca241c4a FAIL (2026-04-05 05:31:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 51.921427988s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.892488307s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:31:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.892513976s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.704882865s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:30:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.704914956s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [..XX....X....XXX.XX........................................X] duration: 5.001095663s trace: https://jaeger.dev.storj.io/trace/6879597f5e6bc99f FAIL (2026-04-05 05:30:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001118557s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [...XX....X....XXX.XX........................................] duration: 3.404786268s trace: https://jaeger.dev.storj.io/trace/7553d45f5b1ac533 FAIL (2026-04-05 05:29:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.404814252s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.6696519s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:29:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.669675375s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 5.861745003s trace: https://jaeger.dev.storj.io/trace/7b008eb7ca241c4a FAIL (2026-04-05 05:29:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.861768077s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.X.X........................................................] duration: 5.594967572s trace: https://jaeger.dev.storj.io/trace/7b008eb7ca241c4a FAIL (2026-04-05 05:29:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.594994274s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [....XX....X....XXX.XX.......................................] duration: 3.535912227s trace: https://jaeger.dev.storj.io/trace/7553d45f5b1ac533 FAIL (2026-04-05 05:29:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.535961292s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.854333904s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:28:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.854360474s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..X.X.......................................................] duration: 5.730925115s trace: https://jaeger.dev.storj.io/trace/7b008eb7ca241c4a FAIL (2026-04-05 05:28:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.730961796s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.819421777s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:28:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.819437988s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.....XX....X....XXX.XX......................................] duration: 3.399232687s trace: https://jaeger.dev.storj.io/trace/7553d45f5b1ac533 FAIL (2026-04-05 05:27:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.399276531s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.816816247s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:27:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.816857146s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...X.X......................................................] duration: 5.575426613s trace: https://jaeger.dev.storj.io/trace/7b008eb7ca241c4a FAIL (2026-04-05 05:27:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.5754658s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [......XX....X....XXX.XX.....................................] duration: 3.354301575s trace: https://jaeger.dev.storj.io/trace/7553d45f5b1ac533 FAIL (2026-04-05 05:27:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.354328898s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.668806481s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:27:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.668833722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.69782619s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:26:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.697854264s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....X.X.....................................................] duration: 5.839274106s trace: https://jaeger.dev.storj.io/trace/7b008eb7ca241c4a FAIL (2026-04-05 05:26:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.839290747s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.232805025s trace: https://jaeger.dev.storj.io/trace/5a5341770822a3cc FAIL (2026-04-05 05:26:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ce_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.192807417 (714h20m27.105145058s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193033373 (714h20m27.105371014s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193198932 (714h20m27.105536564s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193549759 (714h20m27.10588739s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.193572203 (714h20m27.105909834s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194022512 (714h20m27.106360143s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194104560 (714h20m27.106442191s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194389039 (714h20m27.10672667s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194717774 (714h20m27.107055415s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.194730568 (714h20m27.107068199s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195059643 (714h20m27.107397274s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195234200 (714h20m27.107571831s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195436932 (714h20m27.107774563s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:44.195714016 (714h20m27.108051657s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Uploading piece... Apr 5 05:25:45.988724037 (714h20m28.901061668s) [upload=685519]: Next returned nil segment Apr 5 05:25:45.988736951 (714h20m28.901074582s) [upload=685519]: Waiting for error group managing segments... Apr 5 05:25:46.278334571 (714h20m29.190672202s) [noise=true , address=77.90.31.12:4589 , piece_id=5EIZZQVG4OL56UVOWYVJFMCKN57KGCLVPPNVLX7A55CWMO3MG7HQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.281965999 (714h20m29.19430363s) [noise=true , address=45.91.116.7:4012 , piece_id=K3P4CE47CFI7WDCF2IQSMGI4QYO3OAD22IR4L7NM7DDUCUUAFI5A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.292588509 (714h20m29.204926151s) [noise=true , address=45.91.116.14:4196 , piece_id=BYNHXQ7TDLJTVRCQYQEKDUBYKN246C5U3RKYKUUPPYCKFRFWNT6A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.302631332 (714h20m29.214968973s) [noise=true , address=45.91.116.8:4048 , piece_id=GSZIBQCJMZLV67ZMOBKYKDNZL47QOPUYVCWDVWETHFQYSLNVQUHQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:25:46.305482154 (714h20m29.217819785s) [noise=true , address=77.90.31.14:4665 , piece_id=T3IBLI4ZIWVVES66B2R7WRRZP6D5HTHPCAK5BVPYFB5KMUJV2KDA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: <nil> Apr 5 05:26:13.304943733 (714h20m56.217281364s) [noise=true , address=145.79.227.22:4511 , piece_id=FWYY2WKKRPGVKZUCMBVYOWP6DWBRE6QHSWKLYNRS73OWLMG5Q72A, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG, address:145.79.227.22:4511): protocol: read tcp 188.166.225.170:58840->145.79.227.22:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.304978810 (714h20m56.217316442s) [noise=true , address=145.79.227.42:4121 , piece_id=DYXFXHC6IEEUAVY7AQLGV6JFFZFVFRA376KBQVYDHEKWYKQ4GEBA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305052803 (714h20m56.217390434s) [noise=true , address=145.79.227.28:4729 , piece_id=TE3NYZGTDXCUMJ74EZ3K5BCOIDN4S4UHDQOWND2CZDREAXIS7RFQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: read tcp 188.166.225.170:46538->145.79.227.28:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305085256 (714h20m56.217422877s) [noise=true , address=145.79.227.39:4014 , piece_id=EODV6OPZIWAO63WX4FHD3FZVEIFWRS6ZRTAAO7OEHIASYV3X73EQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:145.79.227.39:4014): protocol: read tcp 188.166.225.170:58958->145.79.227.39:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305144721 (714h20m56.217482352s) [noise=true , address=145.79.227.3:4149 , piece_id=CLGAYBSQB26AWGE2XAH4YD25XLZF7I2VOD7BZVTS7MHZG4JIHAWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: read tcp 188.166.225.170:44166->145.79.227.3:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305173626 (714h20m56.217511257s) [noise=true , address=145.79.227.29:4943 , piece_id=57Z3LYZRMAHOHJMZLX65R37RVJ7O2WDVRNBAVV7EMQYWAS7IOLCQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz, address:145.79.227.29:4943): protocol: read tcp 188.166.225.170:45470->145.79.227.29:4943: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305241718 (714h20m56.217579349s) [noise=true , address=145.79.227.32:4833 , piece_id=H2MW3GELCOWYZZPV5D4YWWIGCXVZMCJVFDQHS2ETHQR6HQSQUUIA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: read tcp 188.166.225.170:42334->145.79.227.32:4833: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305260534 (714h20m56.217598165s) [noise=true , address=145.79.227.7:4298 , piece_id=6GXVZOGMMH56AFQ6U7CEBEBT3632GD754NAN2SBN76ZYKOXPOBWQ, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:145.79.227.7:4298): protocol: read tcp 188.166.225.170:38480->145.79.227.7:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305329106 (714h20m56.217666737s) [noise=true , address=145.79.227.23:4545 , piece_id=OS3XJOHYHNBEADQKIAY5EQSNSJEUGELU3OBSQ77QPVG6KT6XHAFA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: read tcp 188.166.225.170:46896->145.79.227.23:4545: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305435973 (714h20m56.217773604s) [noise=true , address=145.79.227.46:4607 , piece_id=GUKCXM3G4EXJ5P2NRQNXHD5QL6RGFUQ6HMN2GNUMIFHLC46U2USA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:1zDoLfA5bQWM7Uf479gfFVjG3yYvxG5EZS5yZSFGTvoFoPYEAo, address:145.79.227.46:4607): protocol: read tcp 188.166.225.170:49286->145.79.227.46:4607: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305531467 (714h20m56.217869098s) [noise=true , address=145.79.227.18:4010 , piece_id=XC2T7S72SFQ2Z45N7VLGY7CAMBLCZ4MOWU3BALSHACYGOPQPTEWA, seg_pos={0 0}, upload=685519]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:145.79.227.18:4010): protocol: read tcp 188.166.225.170:44936->145.79.227.18:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 05:26:13.305549902 (714h20m56.217887534s) [seg_pos={0 0}, upload=685519]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 05:26:13.536348486 (714h20m56.448686117s) [upload=685519]: Done uploading segments... Apr 5 05:26:13.536382360 (714h20m56.448719991s) [upload=685519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.232824443s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)