Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.75199146s FAIL (2025-08-09 11:59:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.752015337s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................X..................X.........] duration: 1.868389538s FAIL (2025-08-09 11:58:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:51:31.707438966 (18h25m26.138519097s) [upload=19243]: Starting upload Aug 9 11:51:31.707508600 (18h25m26.138588731s) [upload=19243]: Done starting upload Aug 9 11:51:31.707610344 (18h25m26.138690465s) [upload=19243]: Uploading segments... Aug 9 11:51:31.707695637 (18h25m26.138775778s) [upload=19243]: Got next segment. Inline: true Aug 9 11:51:31.707703893 (18h25m26.138784055s) [upload=19243]: Waiting for error group managing segments... Aug 9 11:51:31.707813263 (18h25m26.138893404s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:51:31.707823101 (18h25m26.13891722s) [upload=19243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:51:31.707840765 (18h25m26.138920896s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:51:31.925663153 (18h25m26.356743284s) [upload=19243]: Done uploading segments... Aug 9 11:51:31.925680076 (18h25m26.356760207s) [upload=19243]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.868412191s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X............X] duration: 851.933079ms FAIL (2025-08-09 11:58:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:58:42.586754892 (18h32m37.017835023s) [upload=19372]: Starting upload Aug 9 11:58:42.586848722 (18h32m37.017928863s) [upload=19372]: Done starting upload Aug 9 11:58:42.586948633 (18h32m37.018028754s) [upload=19372]: Uploading segments... Aug 9 11:58:42.587092008 (18h32m37.018172139s) [upload=19372]: Got next segment. Inline: false Aug 9 11:58:42.587118157 (18h32m37.018198289s) [upload=19372]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:58:42.587124269 (18h32m37.018204431s) [upload=19372]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:58:42.798975944 (18h32m37.230056075s) [upload=19372]: Next segment err: metaclient: Too Many Requests Aug 9 11:58:42.799010359 (18h32m37.23009049s) [upload=19372]: Done uploading segments... Aug 9 11:58:42.799028504 (18h32m37.230108635s) [upload=19372]: Upload finished. err: metaclient: Too Many Requests Duration: 851.956834ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.773355355s FAIL (2025-08-09 11:58:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.773381826s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.648114626s FAIL (2025-08-09 11:57:52 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.648141285s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X..................X........] duration: 1.849014589s FAIL (2025-08-09 11:57:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:51:31.707438966 (18h25m26.138519097s) [upload=19243]: Starting upload Aug 9 11:51:31.707508600 (18h25m26.138588731s) [upload=19243]: Done starting upload Aug 9 11:51:31.707610344 (18h25m26.138690465s) [upload=19243]: Uploading segments... Aug 9 11:51:31.707695637 (18h25m26.138775778s) [upload=19243]: Got next segment. Inline: true Aug 9 11:51:31.707703893 (18h25m26.138784055s) [upload=19243]: Waiting for error group managing segments... Aug 9 11:51:31.707813263 (18h25m26.138893404s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:51:31.707823101 (18h25m26.13891722s) [upload=19243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:51:31.707840765 (18h25m26.138920896s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:51:31.925663153 (18h25m26.356743284s) [upload=19243]: Done uploading segments... Aug 9 11:51:31.925680076 (18h25m26.356760207s) [upload=19243]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.849035619s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.738546517s FAIL (2025-08-09 11:57:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.738573719s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.78641334s FAIL (2025-08-09 11:56:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.786439579s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.559604287s FAIL (2025-08-09 11:56:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.559629867s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................X..................X.......] duration: 1.847249781s FAIL (2025-08-09 11:56:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:51:31.707438966 (18h25m26.138519097s) [upload=19243]: Starting upload Aug 9 11:51:31.707508600 (18h25m26.138588731s) [upload=19243]: Done starting upload Aug 9 11:51:31.707610344 (18h25m26.138690465s) [upload=19243]: Uploading segments... Aug 9 11:51:31.707695637 (18h25m26.138775778s) [upload=19243]: Got next segment. Inline: true Aug 9 11:51:31.707703893 (18h25m26.138784055s) [upload=19243]: Waiting for error group managing segments... Aug 9 11:51:31.707813263 (18h25m26.138893404s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:51:31.707823101 (18h25m26.13891722s) [upload=19243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:51:31.707840765 (18h25m26.138920896s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:51:31.925663153 (18h25m26.356743284s) [upload=19243]: Done uploading segments... Aug 9 11:51:31.925680076 (18h25m26.356760207s) [upload=19243]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.847276523s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.703115307s FAIL (2025-08-09 11:56:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.703140104s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.605942787s FAIL (2025-08-09 11:56:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.605977153s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................X..................X......] duration: 1.86190812s FAIL (2025-08-09 11:56:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:51:31.707438966 (18h25m26.138519097s) [upload=19243]: Starting upload Aug 9 11:51:31.707508600 (18h25m26.138588731s) [upload=19243]: Done starting upload Aug 9 11:51:31.707610344 (18h25m26.138690465s) [upload=19243]: Uploading segments... Aug 9 11:51:31.707695637 (18h25m26.138775778s) [upload=19243]: Got next segment. Inline: true Aug 9 11:51:31.707703893 (18h25m26.138784055s) [upload=19243]: Waiting for error group managing segments... Aug 9 11:51:31.707813263 (18h25m26.138893404s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:51:31.707823101 (18h25m26.13891722s) [upload=19243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:51:31.707840765 (18h25m26.138920896s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:51:31.925663153 (18h25m26.356743284s) [upload=19243]: Done uploading segments... Aug 9 11:51:31.925680076 (18h25m26.356760207s) [upload=19243]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.861940832s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.698322978s FAIL (2025-08-09 11:55:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.698343566s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.611659845s FAIL (2025-08-09 11:55:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.611689832s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.906945344s FAIL (2025-08-09 11:55:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.906975501s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.907402919s FAIL (2025-08-09 11:55:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.907443196s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X..................X.....] duration: 1.934384724s FAIL (2025-08-09 11:54:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:51:31.707438966 (18h25m26.138519097s) [upload=19243]: Starting upload Aug 9 11:51:31.707508600 (18h25m26.138588731s) [upload=19243]: Done starting upload Aug 9 11:51:31.707610344 (18h25m26.138690465s) [upload=19243]: Uploading segments... Aug 9 11:51:31.707695637 (18h25m26.138775778s) [upload=19243]: Got next segment. Inline: true Aug 9 11:51:31.707703893 (18h25m26.138784055s) [upload=19243]: Waiting for error group managing segments... Aug 9 11:51:31.707813263 (18h25m26.138893404s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:51:31.707823101 (18h25m26.13891722s) [upload=19243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:51:31.707840765 (18h25m26.138920896s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:51:31.925663153 (18h25m26.356743284s) [upload=19243]: Done uploading segments... Aug 9 11:51:31.925680076 (18h25m26.356760207s) [upload=19243]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.934411065s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.735235923s FAIL (2025-08-09 11:54:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.735274617s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X..................X....] duration: 1.869728863s FAIL (2025-08-09 11:54:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:51:31.707438966 (18h25m26.138519097s) [upload=19243]: Starting upload Aug 9 11:51:31.707508600 (18h25m26.138588731s) [upload=19243]: Done starting upload Aug 9 11:51:31.707610344 (18h25m26.138690465s) [upload=19243]: Uploading segments... Aug 9 11:51:31.707695637 (18h25m26.138775778s) [upload=19243]: Got next segment. Inline: true Aug 9 11:51:31.707703893 (18h25m26.138784055s) [upload=19243]: Waiting for error group managing segments... Aug 9 11:51:31.707813263 (18h25m26.138893404s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:51:31.707823101 (18h25m26.13891722s) [upload=19243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:51:31.707840765 (18h25m26.138920896s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:51:31.925663153 (18h25m26.356743284s) [upload=19243]: Done uploading segments... Aug 9 11:51:31.925680076 (18h25m26.356760207s) [upload=19243]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.869754561s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.821163668s FAIL (2025-08-09 11:54:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.82119606s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.780724557s FAIL (2025-08-09 11:53:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.780771087s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.591526082s FAIL (2025-08-09 11:53:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.591552513s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X..................X...] duration: 1.842772366s FAIL (2025-08-09 11:53:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:51:31.707438966 (18h25m26.138519097s) [upload=19243]: Starting upload Aug 9 11:51:31.707508600 (18h25m26.138588731s) [upload=19243]: Done starting upload Aug 9 11:51:31.707610344 (18h25m26.138690465s) [upload=19243]: Uploading segments... Aug 9 11:51:31.707695637 (18h25m26.138775778s) [upload=19243]: Got next segment. Inline: true Aug 9 11:51:31.707703893 (18h25m26.138784055s) [upload=19243]: Waiting for error group managing segments... Aug 9 11:51:31.707813263 (18h25m26.138893404s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:51:31.707823101 (18h25m26.13891722s) [upload=19243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:51:31.707840765 (18h25m26.138920896s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:51:31.925663153 (18h25m26.356743284s) [upload=19243]: Done uploading segments... Aug 9 11:51:31.925680076 (18h25m26.356760207s) [upload=19243]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.842801921s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.55518954s FAIL (2025-08-09 11:53:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.555225619s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................................X..................X..] duration: 1.847733518s FAIL (2025-08-09 11:52:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:51:31.707438966 (18h25m26.138519097s) [upload=19243]: Starting upload Aug 9 11:51:31.707508600 (18h25m26.138588731s) [upload=19243]: Done starting upload Aug 9 11:51:31.707610344 (18h25m26.138690465s) [upload=19243]: Uploading segments... Aug 9 11:51:31.707695637 (18h25m26.138775778s) [upload=19243]: Got next segment. Inline: true Aug 9 11:51:31.707703893 (18h25m26.138784055s) [upload=19243]: Waiting for error group managing segments... Aug 9 11:51:31.707813263 (18h25m26.138893404s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:51:31.707823101 (18h25m26.13891722s) [upload=19243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:51:31.707840765 (18h25m26.138920896s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:51:31.925663153 (18h25m26.356743284s) [upload=19243]: Done uploading segments... Aug 9 11:51:31.925680076 (18h25m26.356760207s) [upload=19243]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.84776017s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.794997204s FAIL (2025-08-09 11:52:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.795015289s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.560056563s FAIL (2025-08-09 11:52:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.560082804s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X..................X.] duration: 1.861441965s FAIL (2025-08-09 11:52:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:51:31.707438966 (18h25m26.138519097s) [upload=19243]: Starting upload Aug 9 11:51:31.707508600 (18h25m26.138588731s) [upload=19243]: Done starting upload Aug 9 11:51:31.707610344 (18h25m26.138690465s) [upload=19243]: Uploading segments... Aug 9 11:51:31.707695637 (18h25m26.138775778s) [upload=19243]: Got next segment. Inline: true Aug 9 11:51:31.707703893 (18h25m26.138784055s) [upload=19243]: Waiting for error group managing segments... Aug 9 11:51:31.707813263 (18h25m26.138893404s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:51:31.707823101 (18h25m26.13891722s) [upload=19243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:51:31.707840765 (18h25m26.138920896s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:51:31.925663153 (18h25m26.356743284s) [upload=19243]: Done uploading segments... Aug 9 11:51:31.925680076 (18h25m26.356760207s) [upload=19243]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.861496981s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.669992504s FAIL (2025-08-09 11:52:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.670026948s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.006086371s FAIL (2025-08-09 11:51:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.006104737s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.594292647s FAIL (2025-08-09 11:51:44 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.594317113s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................X..................X] duration: 876.428375ms FAIL (2025-08-09 11:51:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 11:51:31.707438966 (18h25m26.138519097s) [upload=19243]: Starting upload Aug 9 11:51:31.707508600 (18h25m26.138588731s) [upload=19243]: Done starting upload Aug 9 11:51:31.707610344 (18h25m26.138690465s) [upload=19243]: Uploading segments... Aug 9 11:51:31.707695637 (18h25m26.138775778s) [upload=19243]: Got next segment. Inline: true Aug 9 11:51:31.707703893 (18h25m26.138784055s) [upload=19243]: Waiting for error group managing segments... Aug 9 11:51:31.707813263 (18h25m26.138893404s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:51:31.707823101 (18h25m26.13891722s) [upload=19243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:51:31.707840765 (18h25m26.138920896s) [upload=19243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:51:31.925663153 (18h25m26.356743284s) [upload=19243]: Done uploading segments... Aug 9 11:51:31.925680076 (18h25m26.356760207s) [upload=19243]: Upload finished. err: metaclient: Too Many Requests Duration:* 876.445918ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.918135398s FAIL (2025-08-09 11:51:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 11:36:16.645645963 (18h10m11.076726094s) [upload=18981]: Starting upload Aug 9 11:36:16.645692913 (18h10m11.076773034s) [upload=18981]: Done starting upload Aug 9 11:36:16.645742107 (18h10m11.076822238s) [upload=18981]: Uploading segments... Aug 9 11:36:16.645832941 (18h10m11.076913112s) [upload=18981]: Got next segment. Inline: true Aug 9 11:36:16.645840495 (18h10m11.076920646s) [upload=18981]: Waiting for error group managing segments... Aug 9 11:36:16.645889229 (18h10m11.07696936s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.645894428 (18h10m11.076974579s) [upload=18981]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.645898245 (18h10m11.076978387s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.943127530 (18h10m11.374207661s) [upload=18981]: Done uploading segments... Aug 9 11:36:16.943148580 (18h10m11.374228731s) [upload=18981]: Upload finished. err: <nil> Duration:* 1.918161798s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.646789788s FAIL (2025-08-09 11:51:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 5.646816761s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.876018572s FAIL (2025-08-09 11:51:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.876041487s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.853033478s FAIL (2025-08-09 11:50:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 11:36:16.645645963 (18h10m11.076726094s) [upload=18981]: Starting upload Aug 9 11:36:16.645692913 (18h10m11.076773034s) [upload=18981]: Done starting upload Aug 9 11:36:16.645742107 (18h10m11.076822238s) [upload=18981]: Uploading segments... Aug 9 11:36:16.645832941 (18h10m11.076913112s) [upload=18981]: Got next segment. Inline: true Aug 9 11:36:16.645840495 (18h10m11.076920646s) [upload=18981]: Waiting for error group managing segments... Aug 9 11:36:16.645889229 (18h10m11.07696936s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.645894428 (18h10m11.076974579s) [upload=18981]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.645898245 (18h10m11.076978387s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.943127530 (18h10m11.374207661s) [upload=18981]: Done uploading segments... Aug 9 11:36:16.943148580 (18h10m11.374228731s) [upload=18981]: Upload finished. err: <nil> Duration:* 1.853058296s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 4.229565944s FAIL (2025-08-09 11:50:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 11:50:42.980337174 (18h24m37.411417305s) [upload=19229]: Starting upload Aug 9 11:50:42.980392921 (18h24m37.411473052s) [upload=19229]: Done starting upload Aug 9 11:50:42.980568757 (18h24m37.411648888s) [upload=19229]: Uploading segments... Aug 9 11:50:42.980666124 (18h24m37.411746255s) [upload=19229]: Got next segment. Inline: false Aug 9 11:50:42.980709998 (18h24m37.411790129s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:50:42.980714897 (18h24m37.411795028s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:50:43.438740956 (18h24m37.869821087s) [seg_pos={0 0}, upload=19229]: Begin upload segment... Aug 9 11:50:43.438790972 (18h24m37.869871103s) [seg_pos={0 0}, upload=19229]: Done begin upload segment. Aug 9 11:50:43.438827331 (18h24m37.869907452s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.438888128 (18h24m37.869968269s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439234410 (18h24m37.870314541s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439522211 (18h24m37.870602342s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439534234 (18h24m37.870614365s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439570042 (18h24m37.870650163s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439842814 (18h24m37.870922945s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.439991228 (18h24m37.871071359s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440093684 (18h24m37.871173815s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440280241 (18h24m37.871360362s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440415430 (18h24m37.871495561s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440708981 (18h24m37.871789113s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.440782021 (18h24m37.871862152s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441003435 (18h24m37.872083566s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441062668 (18h24m37.872142799s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441249124 (18h24m37.872329245s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441369024 (18h24m37.872449155s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:43.441492139 (18h24m37.87257227s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Uploading piece... Aug 9 11:50:44.565549483 (18h24m38.996629614s) [upload=19229]: Next returned nil segment Aug 9 11:50:44.565566294 (18h24m38.996646425s) [upload=19229]: Waiting for error group managing segments... Aug 9 11:50:44.852354444 (18h24m39.283434575s) [noise=true , address=64.49.13.154:4927 , piece_id=KCVYUB7VKEBS3TB73KLT3YM2C6J6ZVRPR2I5FNHJ3AJVL3GOY2IQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.902059725 (18h24m39.333139857s) [noise=true , address=64.49.13.139:4479 , piece_id=LE6SSWZCPZ5DNXLKKIXUJJICBSMOPT6RCDYJDJRGM5E23FQWCKUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:44.921191681 (18h24m39.352271802s) [noise=true , address=64.49.13.26:4946 , piece_id=XMSO3A34WRZFKPAVHU7ZC4YTOL4AVPDOTVBK6ZWZ556SDHWXLRTQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.371863388 (18h24m39.802943519s) [noise=true , address=45.91.116.12:4234 , piece_id=T5RHTHS4NZXVTY6P5WR3ZAZUMI45UVPUGAAY4SSX3ROWN7ZLCRAA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.376719068 (18h24m39.8077992s) [noise=true , address=104.36.82.18:4367 , piece_id=RPEPXROMT7P6COJFQOYUS4IBUKW2EACDX4AFXYZ2E4C7LDD737NQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.392100417 (18h24m39.823180548s) [noise=true , address=104.36.82.36:4630 , piece_id=77ODIX3TNUIGV7PDA3UQ7YJPZMPJU2VCG3RYLUDMAMR3DXUGCVMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.395413386 (18h24m39.826493517s) [noise=true , address=104.36.82.44:4888 , piece_id=NBLTVQFRFADE7CEKYLHHELIO7JKJGSVXIQILPGBLNGCF6LBGCRPQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.399980545 (18h24m39.831060666s) [noise=true , address=45.91.116.5:4320 , piece_id=2NUUA26OOI4ME3RTLLWIWTOXRNHUQCM3FZSFWB2VCUIUW6MT5OXA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.404941857 (18h24m39.836021988s) [noise=true , address=104.36.82.47:4543 , piece_id=IXICQPYSZ6T2UKYF5EEGGB27WCDCJP2U6M52RY7WD2G4O67DGBMA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.453041415 (18h24m39.884121546s) [noise=true , address=104.36.82.60:4364 , piece_id=DV52WN3UVN53JP7BIAXHH7X2ZYGWW6BA65D7DNTIHHVS7NZUMIZA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.468341980 (18h24m39.899422111s) [noise=true , address=104.36.82.34:4551 , piece_id=4QK6N4H6YQHBCSHZ6QBA7LQRGVVXAJPKYLM2P3MCLVMO7Z3TAS3Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.473371052 (18h24m39.904451183s) [noise=true , address=104.36.82.38:4719 , piece_id=PFEXW34DREFSVNL3XKE6V2F5GTMWFBK5VASBXEVZJ7O2PDWVNSJQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.482757431 (18h24m39.913837562s) [noise=true , address=104.36.82.60:4338 , piece_id=6PZ5WO3WHBFVN6HS43EMETH6NGMCVDTPOQYSPVDACAFA6FTNPECQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490560320 (18h24m39.921640451s) [noise=true , address=104.36.82.14:4230 , piece_id=PPNIAHYAWZMI65P62RTC2WPFKYW2IP2EOU7ZQZOUVWVLYM2RHRUA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: <nil> Aug 9 11:50:45.490575379 (18h24m39.92165551s) [seg_pos={0 0}, upload=19229]: Segment reached optimal threshold of 14 pieces. Aug 9 11:50:45.490695979 (18h24m39.92177616s) [noise=true , address=104.36.82.56:4244 , piece_id=K7EU7NCQ6CEFO5HQ6XVRFIDGUZSI6RSRNJ4H2MBC2ETUS5X3OJ6A, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490765072 (18h24m39.921845203s) [noise=true , address=104.36.82.52:4100 , piece_id=P6MEMF6QTU4ECUHKPVCTJ46MJUBGU26QVIGOA7J6IFDI4PISJANQ, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JQ78Lg28PborUyvkyLJb5RikJ8zJSAT5GkpYSsZLc6HRmkhyw): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490825287 (18h24m39.921905418s) [noise=true , address=104.36.82.23:4452 , piece_id=A6THRC37JB3VSHG5XO4A4PDGUUXDE3NUTT6TAZGQH27A4AA26IKA, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490846628 (18h24m39.921926759s) [noise=true , address=104.36.82.61:4401 , piece_id=2E327EBS52BHWGME3QCDFRAOJYDPHVBWALJAIZ4QEGWO63J4N54Q, seg_pos={0 0}, upload=19229]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY): protocol: expected piece hash; context canceled; EOF Aug 9 11:50:45.490884200 (18h24m39.921964331s) [seg_pos={0 0}, upload=19229]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:50:45.490957009 (18h24m39.92203714s) [upload=19229]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:50:45.490960907 (18h24m39.922041038s) [upload=19229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:50:45.826729161 (18h24m40.257809282s) [upload=19229]: Done uploading segments... Aug 9 11:50:45.826748577 (18h24m40.257828708s) [upload=19229]: Upload finished. err: <nil> Duration: 4.22959071s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.739824751s FAIL (2025-08-09 11:50:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.739849982s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.862963116s FAIL (2025-08-09 11:50:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 11:36:16.645645963 (18h10m11.076726094s) [upload=18981]: Starting upload Aug 9 11:36:16.645692913 (18h10m11.076773034s) [upload=18981]: Done starting upload Aug 9 11:36:16.645742107 (18h10m11.076822238s) [upload=18981]: Uploading segments... Aug 9 11:36:16.645832941 (18h10m11.076913112s) [upload=18981]: Got next segment. Inline: true Aug 9 11:36:16.645840495 (18h10m11.076920646s) [upload=18981]: Waiting for error group managing segments... Aug 9 11:36:16.645889229 (18h10m11.07696936s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.645894428 (18h10m11.076974579s) [upload=18981]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.645898245 (18h10m11.076978387s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.943127530 (18h10m11.374207661s) [upload=18981]: Done uploading segments... Aug 9 11:36:16.943148580 (18h10m11.374228731s) [upload=18981]: Upload finished. err: <nil> Duration:* 1.862983985s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.808530084s FAIL (2025-08-09 11:49:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.808556606s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 3.509468444s FAIL (2025-08-09 11:49:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 11:36:16.645645963 (18h10m11.076726094s) [upload=18981]: Starting upload Aug 9 11:36:16.645692913 (18h10m11.076773034s) [upload=18981]: Done starting upload Aug 9 11:36:16.645742107 (18h10m11.076822238s) [upload=18981]: Uploading segments... Aug 9 11:36:16.645832941 (18h10m11.076913112s) [upload=18981]: Got next segment. Inline: true Aug 9 11:36:16.645840495 (18h10m11.076920646s) [upload=18981]: Waiting for error group managing segments... Aug 9 11:36:16.645889229 (18h10m11.07696936s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.645894428 (18h10m11.076974579s) [upload=18981]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.645898245 (18h10m11.076978387s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.943127530 (18h10m11.374207661s) [upload=18981]: Done uploading segments... Aug 9 11:36:16.943148580 (18h10m11.374228731s) [upload=18981]: Upload finished. err: <nil> Duration:* 3.509502671s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.798938189s FAIL (2025-08-09 11:48:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.798962377s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.887832299s FAIL (2025-08-09 11:48:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 11:36:16.645645963 (18h10m11.076726094s) [upload=18981]: Starting upload Aug 9 11:36:16.645692913 (18h10m11.076773034s) [upload=18981]: Done starting upload Aug 9 11:36:16.645742107 (18h10m11.076822238s) [upload=18981]: Uploading segments... Aug 9 11:36:16.645832941 (18h10m11.076913112s) [upload=18981]: Got next segment. Inline: true Aug 9 11:36:16.645840495 (18h10m11.076920646s) [upload=18981]: Waiting for error group managing segments... Aug 9 11:36:16.645889229 (18h10m11.07696936s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.645894428 (18h10m11.076974579s) [upload=18981]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.645898245 (18h10m11.076978387s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.943127530 (18h10m11.374207661s) [upload=18981]: Done uploading segments... Aug 9 11:36:16.943148580 (18h10m11.374228731s) [upload=18981]: Upload finished. err: <nil> Duration:* 1.887865222s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 13.09336655s FAIL (2025-08-09 11:47:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 13.093400735s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.896408148s FAIL (2025-08-09 11:47:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 11:36:16.645645963 (18h10m11.076726094s) [upload=18981]: Starting upload Aug 9 11:36:16.645692913 (18h10m11.076773034s) [upload=18981]: Done starting upload Aug 9 11:36:16.645742107 (18h10m11.076822238s) [upload=18981]: Uploading segments... Aug 9 11:36:16.645832941 (18h10m11.076913112s) [upload=18981]: Got next segment. Inline: true Aug 9 11:36:16.645840495 (18h10m11.076920646s) [upload=18981]: Waiting for error group managing segments... Aug 9 11:36:16.645889229 (18h10m11.07696936s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.645894428 (18h10m11.076974579s) [upload=18981]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.645898245 (18h10m11.076978387s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.943127530 (18h10m11.374207661s) [upload=18981]: Done uploading segments... Aug 9 11:36:16.943148580 (18h10m11.374228731s) [upload=18981]: Upload finished. err: <nil> Duration:* 1.896424138s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.973821376s FAIL (2025-08-09 11:47:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005421531 (18h11m19.436501662s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005614329 (18h11m19.43669446s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.005727536 (18h11m19.436807667s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006281517 (18h11m19.437361648s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006632558 (18h11m19.437712679s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.006996264 (18h11m19.438076395s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007245231 (18h11m19.438325362s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007475481 (18h11m19.438555612s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:25.007873984 (18h11m19.438954115s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Uploading piece... Aug 9 11:37:26.733124951 (18h11m21.164205082s) [upload=18998]: Next returned nil segment Aug 9 11:37:26.733142194 (18h11m21.164222325s) [upload=18998]: Waiting for error group managing segments... Aug 9 11:37:27.100757697 (18h11m21.531837848s) [noise=true , address=104.36.82.14:4222 , piece_id=Z4X6RJ5HBAYGUNKT6DXWSS2TEG2OUFPMH3NWYGA7D26XIFH46UAA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.106781913 (18h11m21.537862064s) [noise=true , address=104.36.82.7:4912 , piece_id=MRA5X6N6MDQ2TADHZNIB6LBP5VQHFEGSYM3YNNZWNM2N4UU7JTCQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.136679105 (18h11m21.567759236s) [noise=true , address=104.36.82.5:4838 , piece_id=T3WCWCLB4WAXQY3S6VCFHLKYK6IY2Q6P2776CLGYS4QOFTXMYVEQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.139126447 (18h11m21.570206578s) [noise=true , address=104.36.82.3:4765 , piece_id=HBLTTPZYK5ZXNG2XV53QVJA3NPX3YWKEAAAFVMDXUO6BLR3QKCWQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:27.183514474 (18h11m21.614594606s) [noise=true , address=104.36.82.7:4909 , piece_id=OGG5ZCXSLIM5YKMM7K6KLKLWGOTX376DUTDT3567OB3AJIBPBKDQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:53.489259032 (18h11m47.920339163s) [noise=true , address=45.91.116.13:4155 , piece_id=K4CGD4OCR2B6ZWAZHB5MDBECVJB3AC2HDXS2TUVAVGIWQYJEARLA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: <nil> Aug 9 11:37:54.124525138 (18h11m48.555605259s) [noise=true , address=45.91.116.7:4010 , piece_id=5G3O552GD5XO7SNIHST623QJBWJF2C7B6EN7ISGHHY3FYV6QL2DQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54960->45.91.116.7:4010: use of closed network connection Aug 9 11:37:54.124581035 (18h11m48.555661166s) [noise=true , address=77.90.31.10:4520 , piece_id=YRBBWM54FGU3B74JQ7EOAO7VSGVFXAVT4SNZ6KN3IQFIHIEVV5HA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39560->77.90.31.10:4520: use of closed network connection Aug 9 11:37:54.124615480 (18h11m48.555695652s) [noise=true , address=45.91.116.12:4225 , piece_id=VFK2SMVTBT76D7YLQWGRA6TF47JD3B7J7F5NFR4E2MQVNGNHYTAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38054->45.91.116.12:4225: use of closed network connection Aug 9 11:37:54.124643875 (18h11m48.555724016s) [noise=true , address=45.91.116.4:4940 , piece_id=M33ZGUCEF23HDGQZE5F5KLWQ43NHC7E7LKOVVGQ6P6TM2HFGB4WQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40630->45.91.116.4:4940: use of closed network connection Aug 9 11:37:54.124693159 (18h11m48.55577335s) [noise=true , address=45.91.116.14:4189 , piece_id=XVQMTYHB6H3IKSERYT2BEB4NP42KV6FRE6J2MKKU5WEOCVBHYIWA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37650->45.91.116.14:4189: use of closed network connection Aug 9 11:37:54.124771740 (18h11m48.555851871s) [noise=true , address=77.90.31.18:4806 , piece_id=QCURQYEJL22LBT2YFBR62RIHH7VNSGL7GERP66GKHXK3JP5WLVZQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55536->77.90.31.18:4806: use of closed network connection Aug 9 11:37:54.124817217 (18h11m48.555897348s) [noise=true , address=77.90.31.15:4697 , piece_id=Q26SRGZO2YBJWPDIG4U422EICE6XEXPLHFHXLKFUBKMMJAZ4W7UQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49618->77.90.31.15:4697: use of closed network connection Aug 9 11:37:54.124833387 (18h11m48.555913559s) [noise=true , address=45.91.116.6:4973 , piece_id=MHDNXTVKFMLRKGS7JIPL4U5VQMBXGP4EYM7JEGPSK4U3GIJ5JEHQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59438->45.91.116.6:4973: use of closed network connection Aug 9 11:37:54.124840752 (18h11m48.555920883s) [noise=true , address=77.90.31.17:4772 , piece_id=MGZPHOODRIOTGABQJHNIP5FGF4XN6A723ZHUBKI6KNAJDKFWWOJA, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42704->77.90.31.17:4772: use of closed network connection Aug 9 11:37:54.124845251 (18h11m48.555925382s) [noise=true , address=45.91.116.8:4047 , piece_id=B27B2QKZXICP4DHTDTCYSZZ7RN2PPCMKSRQBKRK5JJ4BE4RNWQAQ, seg_pos={0 0}, upload=18998]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40794->45.91.116.8:4047: use of closed network connection Aug 9 11:37:54.124866321 (18h11m48.555946492s) [seg_pos={0 0}, upload=18998]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:54.353990811 (18h11m48.785070942s) [upload=18998]: Done uploading segments... Aug 9 11:37:54.354028673 (18h11m48.785108804s) [upload=18998]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.97384951s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.696338392s FAIL (2025-08-09 11:46:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 3ac1058083644ad645d645969299aef4580161f690b8e8c4752d84e6a75f8af6 != before 4e65aa7fd50d10320a1086a4d09ba7a03e40c2384a6acd458895fc2e7005cc59 ======LOGS====== Aug 9 11:00:27.649216897 (17h34m22.080297028s) [upload=18353]: Starting upload Aug 9 11:00:27.649258687 (17h34m22.080338818s) [upload=18353]: Done starting upload Aug 9 11:00:27.649474761 (17h34m22.080554922s) [upload=18353]: Uploading segments... Aug 9 11:00:27.649578108 (17h34m22.08065826s) [upload=18353]: Got next segment. Inline: false Aug 9 11:00:27.649615870 (17h34m22.080695991s) [upload=18353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:00:27.649621872 (17h34m22.080702003s) [upload=18353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:00:28.101416518 (17h34m22.532496659s) [seg_pos={0 0}, upload=18353]: Begin upload segment... Aug 9 11:00:28.101481252 (17h34m22.532561383s) [seg_pos={0 0}, upload=18353]: Done begin upload segment. Aug 9 11:00:28.101512832 (17h34m22.532592963s) [noise=true , address=104.36.82.59:4314 , piece_id=ZSDJHG3C5KFT6UV2U5VPOXSVTH2QYNARAO3GV26DV43NBKGWQ7RQ, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.101608826 (17h34m22.532688967s) [noise=true , address=104.36.82.8:4987 , piece_id=GHKDW3GCYWTH57VXGMGTQNNH45M6QVBARAZ5VE7VZVDM7UHSFX5A, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.101934789 (17h34m22.53301492s) [noise=true , address=104.36.82.38:4697 , piece_id=XLBY3P5WRMWP3O2LEZI2UYWCCVR2ASPLFLPUN2DSKJXTTULARGMQ, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.102201600 (17h34m22.533281731s) [noise=true , address=104.36.82.34:4580 , piece_id=SZ3D6NVS7QDJFVQYVRBOUXKG65EFPX3Z5K5SH7LXQY7SEDFYC2XA, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.102244472 (17h34m22.533324603s) [noise=true , address=104.36.82.14:4237 , piece_id=RPO2UBXUXEWXLG3Y6KYCMOO4U4GU3T6WRJ6QKRTXIGRBVAPY4OGA, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.102510651 (17h34m22.533590792s) [noise=true , address=104.36.82.60:4367 , piece_id=WNTFJKAIOHHVHZNREAURKLOGWUVIS2OQZGIIFIVNSXURS6BGGQFA, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.102659626 (17h34m22.533739757s) [noise=true , address=77.90.31.15:4705 , piece_id=NBP7PIDX5Q3OR5OWLAKWNHV3GEXPTWKRLKXRUWZXKWFXFGC37JPA, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.102788303 (17h34m22.533868434s) [noise=true , address=45.91.116.13:4187 , piece_id=NEWJSITVRBKSTZPFGP2OFXOWBUN7OHMU5FFTRZRNIDVZ63GLXSEA, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.103004816 (17h34m22.534084947s) [noise=true , address=104.36.82.44:4897 , piece_id=Z3JHTV2RXJNQPHPAGXMQEMPPMUDPC24DXDOIXXIZNWNLKTFYXJZQ, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.103016750 (17h34m22.534096881s) [noise=true , address=104.36.82.22:4520 , piece_id=RCYXJ7LDHRRQYTB5672WD3YVNDN7IEKZQQM7YMZDMK225E3CSVOQ, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.103247010 (17h34m22.534327141s) [noise=true , address=104.36.82.62:4439 , piece_id=RXB7U7G4P7YUBYV5YBIEXSGIQRSA63XETSH46B42SXYOQQEBZVKA, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.103377540 (17h34m22.534457671s) [noise=true , address=104.36.82.35:4590 , piece_id=CQXVORKT6IA7SBFSBJWTLSN4L7WVWTMU5BNNW25V2MXHLCIXIHIA, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.103493201 (17h34m22.534573332s) [noise=true , address=104.36.82.12:4175 , piece_id=6OFNAUH6H42VDWHGTYCMRBUL37QR6UZM4W6GQX27IEJATYZLTYZA, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.103697011 (17h34m22.534777143s) [noise=true , address=104.36.82.46:4488 , piece_id=ZEEQDE334NA2YQEVR5YZ6O6QHE36L7GJHCA4GHU2XPGWZ4GFI5HQ, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.103711669 (17h34m22.53479181s) [noise=true , address=64.49.13.154:4933 , piece_id=GFMX7MSCDHULD3WLVS32PUKZU3LY7PK3DQXO4EJ4PB4BVXVJBNXQ, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.104002656 (17h34m22.535082797s) [noise=true , address=104.36.82.19:4407 , piece_id=IVSO2PJPSDC65PY4MECXCDQTWTCVA7MDTEE2QLJLEOT5TBHW473Q, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.106216291 (17h34m22.537296422s) [noise=true , address=104.36.82.44:4880 , piece_id=VQHMO4UGE44V4KPQLYPARQTGY3W2GL2OMT6P6GPCH4EAVCBAFJJA, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:28.106499442 (17h34m22.537579574s) [noise=true , address=104.36.82.16:4315 , piece_id=BWHAIBBKL2URMBWFRKBKIK3SBIGDWNPGVX6M4JYJPRC3554XFETA, seg_pos={0 0}, upload=18353]: Uploading piece... Aug 9 11:00:29.278465108 (17h34m23.709545259s) [upload=18353]: Next returned nil segment Aug 9 11:00:29.278491478 (17h34m23.709571609s) [upload=18353]: Waiting for error group managing segments... Aug 9 11:00:29.574413815 (17h34m24.005493946s) [noise=true , address=64.49.13.154:4933 , piece_id=GFMX7MSCDHULD3WLVS32PUKZU3LY7PK3DQXO4EJ4PB4BVXVJBNXQ, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.023351305 (17h34m24.454431436s) [noise=true , address=45.91.116.13:4187 , piece_id=NEWJSITVRBKSTZPFGP2OFXOWBUN7OHMU5FFTRZRNIDVZ63GLXSEA, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.052186965 (17h34m24.483267096s) [noise=true , address=104.36.82.60:4367 , piece_id=WNTFJKAIOHHVHZNREAURKLOGWUVIS2OQZGIIFIVNSXURS6BGGQFA, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.059968333 (17h34m24.491048464s) [noise=true , address=104.36.82.35:4590 , piece_id=CQXVORKT6IA7SBFSBJWTLSN4L7WVWTMU5BNNW25V2MXHLCIXIHIA, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.059997509 (17h34m24.49107764s) [noise=true , address=104.36.82.12:4175 , piece_id=6OFNAUH6H42VDWHGTYCMRBUL37QR6UZM4W6GQX27IEJATYZLTYZA, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.065290015 (17h34m24.496370156s) [noise=true , address=104.36.82.59:4314 , piece_id=ZSDJHG3C5KFT6UV2U5VPOXSVTH2QYNARAO3GV26DV43NBKGWQ7RQ, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.069464021 (17h34m24.500544182s) [noise=true , address=104.36.82.8:4987 , piece_id=GHKDW3GCYWTH57VXGMGTQNNH45M6QVBARAZ5VE7VZVDM7UHSFX5A, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.073204137 (17h34m24.504284268s) [noise=true , address=77.90.31.15:4705 , piece_id=NBP7PIDX5Q3OR5OWLAKWNHV3GEXPTWKRLKXRUWZXKWFXFGC37JPA, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.108262184 (17h34m24.539342345s) [noise=true , address=104.36.82.38:4697 , piece_id=XLBY3P5WRMWP3O2LEZI2UYWCCVR2ASPLFLPUN2DSKJXTTULARGMQ, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.109483421 (17h34m24.540563562s) [noise=true , address=104.36.82.19:4407 , piece_id=IVSO2PJPSDC65PY4MECXCDQTWTCVA7MDTEE2QLJLEOT5TBHW473Q, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.120388194 (17h34m24.551468325s) [noise=true , address=104.36.82.14:4237 , piece_id=RPO2UBXUXEWXLG3Y6KYCMOO4U4GU3T6WRJ6QKRTXIGRBVAPY4OGA, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.128026749 (17h34m24.55910688s) [noise=true , address=104.36.82.34:4580 , piece_id=SZ3D6NVS7QDJFVQYVRBOUXKG65EFPX3Z5K5SH7LXQY7SEDFYC2XA, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.130317512 (17h34m24.561397643s) [noise=true , address=104.36.82.46:4488 , piece_id=ZEEQDE334NA2YQEVR5YZ6O6QHE36L7GJHCA4GHU2XPGWZ4GFI5HQ, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.130733388 (17h34m24.561813519s) [noise=true , address=104.36.82.44:4880 , piece_id=VQHMO4UGE44V4KPQLYPARQTGY3W2GL2OMT6P6GPCH4EAVCBAFJJA, seg_pos={0 0}, upload=18353]: Done uploading piece. err: <nil> Aug 9 11:00:30.130742947 (17h34m24.561823078s) [seg_pos={0 0}, upload=18353]: Segment reached optimal threshold of 14 pieces. Aug 9 11:00:30.130844139 (17h34m24.561924271s) [noise=true , address=104.36.82.44:4897 , piece_id=Z3JHTV2RXJNQPHPAGXMQEMPPMUDPC24DXDOIXXIZNWNLKTFYXJZQ, seg_pos={0 0}, upload=18353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 11:00:30.131970324 (17h34m24.563050455s) [noise=true , address=104.36.82.22:4520 , piece_id=RCYXJ7LDHRRQYTB5672WD3YVNDN7IEKZQQM7YMZDMK225E3CSVOQ, seg_pos={0 0}, upload=18353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tf69f5XQmK4E6Bum2DwaCgSfexAh2v7ZxN9KjfzSCAUN5bTkH): protocol: expected piece hash; context canceled; EOF Aug 9 11:00:30.132070927 (17h34m24.563151058s) [noise=true , address=104.36.82.62:4439 , piece_id=RXB7U7G4P7YUBYV5YBIEXSGIQRSA63XETSH46B42SXYOQQEBZVKA, seg_pos={0 0}, upload=18353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HRW3L15ka62qA7mWmEJaJ2ydTA5f8sVY1iL3Fn1R7ubisPF3W): protocol: expected piece hash; context canceled; EOF Aug 9 11:00:30.132135260 (17h34m24.563215391s) [noise=true , address=104.36.82.16:4315 , piece_id=BWHAIBBKL2URMBWFRKBKIK3SBIGDWNPGVX6M4JYJPRC3554XFETA, seg_pos={0 0}, upload=18353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: expected piece hash; context canceled; EOF Aug 9 11:00:30.132145500 (17h34m24.563225631s) [seg_pos={0 0}, upload=18353]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 11:00:30.132215383 (17h34m24.563295514s) [upload=18353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:00:30.132219381 (17h34m24.563299512s) [upload=18353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:00:30.561346767 (17h34m24.992426898s) [upload=18353]: Done uploading segments... Aug 9 11:00:30.561365874 (17h34m24.992446005s) [upload=18353]: Upload finished. err: <nil> Duration: 5.696366375s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.843057052s FAIL (2025-08-09 11:46:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 11:36:16.645645963 (18h10m11.076726094s) [upload=18981]: Starting upload Aug 9 11:36:16.645692913 (18h10m11.076773034s) [upload=18981]: Done starting upload Aug 9 11:36:16.645742107 (18h10m11.076822238s) [upload=18981]: Uploading segments... Aug 9 11:36:16.645832941 (18h10m11.076913112s) [upload=18981]: Got next segment. Inline: true Aug 9 11:36:16.645840495 (18h10m11.076920646s) [upload=18981]: Waiting for error group managing segments... Aug 9 11:36:16.645889229 (18h10m11.07696936s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.645894428 (18h10m11.076974579s) [upload=18981]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.645898245 (18h10m11.076978387s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.943127530 (18h10m11.374207661s) [upload=18981]: Done uploading segments... Aug 9 11:36:16.943148580 (18h10m11.374228731s) [upload=18981]: Upload finished. err: <nil> Duration:* 1.843084764s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.837715855s FAIL (2025-08-09 11:46:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 11:36:16.645645963 (18h10m11.076726094s) [upload=18981]: Starting upload Aug 9 11:36:16.645692913 (18h10m11.076773034s) [upload=18981]: Done starting upload Aug 9 11:36:16.645742107 (18h10m11.076822238s) [upload=18981]: Uploading segments... Aug 9 11:36:16.645832941 (18h10m11.076913112s) [upload=18981]: Got next segment. Inline: true Aug 9 11:36:16.645840495 (18h10m11.076920646s) [upload=18981]: Waiting for error group managing segments... Aug 9 11:36:16.645889229 (18h10m11.07696936s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.645894428 (18h10m11.076974579s) [upload=18981]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.645898245 (18h10m11.076978387s) [upload=18981]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.943127530 (18h10m11.374207661s) [upload=18981]: Done uploading segments... Aug 9 11:36:16.943148580 (18h10m11.374228731s) [upload=18981]: Upload finished. err: <nil> Duration:* 1.837747816s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)