Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 6.613577495s FAIL (2025-08-17 04:47:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.613599768s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................XXXX..................................] duration: 5.717570685s FAIL (2025-08-17 04:47:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.717595633s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................................X.....X.........] duration: 1.847516441s FAIL (2025-08-17 04:47:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.847540547s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................XXXX.................................] duration: 6.165148071s FAIL (2025-08-17 04:47:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.165175762s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................XXXX................................] duration: 5.979671498s FAIL (2025-08-17 04:47:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.979701494s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................XXX..................................] duration: 5.769979681s FAIL (2025-08-17 04:47:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.770006904s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [.................X..........XXXX.X.X.......X...X............] duration: 5.777166493s FAIL (2025-08-17 04:46:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 17 04:37:54.182428668 (203h11m48.613508799s) [upload=211361]: Starting upload Aug 17 04:37:54.182474656 (203h11m48.613554777s) [upload=211361]: Done starting upload Aug 17 04:37:54.182665000 (203h11m48.613745131s) [upload=211361]: Uploading segments... Aug 17 04:37:54.182728421 (203h11m48.613808552s) [upload=211361]: Got next segment. Inline: false Aug 17 04:37:54.182758048 (203h11m48.613838179s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:37:54.182762306 (203h11m48.613842437s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:37:54.737160222 (203h11m49.168240353s) [seg_pos={0 0}, upload=211361]: Begin upload segment... Aug 17 04:37:54.737203494 (203h11m49.168283625s) [seg_pos={0 0}, upload=211361]: Done begin upload segment. Aug 17 04:37:54.737224945 (203h11m49.168305076s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737311591 (203h11m49.168391722s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737638917 (203h11m49.168719048s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737939432 (203h11m49.169019563s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737952067 (203h11m49.169032198s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738229999 (203h11m49.16931013s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738324038 (203h11m49.169404169s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738511367 (203h11m49.169591488s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738655813 (203h11m49.169735954s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738747769 (203h11m49.1698279s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738971367 (203h11m49.170051498s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738975705 (203h11m49.170055836s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739194995 (203h11m49.170275126s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739278324 (203h11m49.170358445s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739421147 (203h11m49.170501278s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739561055 (203h11m49.170641186s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739643824 (203h11m49.170723955s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739852683 (203h11m49.170932814s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:55.861243533 (203h11m50.292323664s) [upload=211361]: Next returned nil segment Aug 17 04:37:55.861256768 (203h11m50.292336899s) [upload=211361]: Waiting for error group managing segments... Aug 17 04:37:56.165335502 (203h11m50.596415633s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.263548583 (203h11m50.694628715s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.569392606 (203h11m51.000472737s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.620901052 (203h11m51.051981183s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.639713969 (203h11m51.0707941s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.649068307 (203h11m51.080148438s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.701677831 (203h11m51.132757962s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.719713991 (203h11m51.150794132s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.730632311 (203h11m51.161712432s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.761149974 (203h11m51.192230105s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.775738719 (203h11m51.20681885s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.779316274 (203h11m51.210396405s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.802147800 (203h11m51.233227961s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832614886 (203h11m51.263695017s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832646346 (203h11m51.263726467s) [seg_pos={0 0}, upload=211361]: Segment reached optimal threshold of 14 pieces. Aug 17 04:37:56.832765443 (203h11m51.263845574s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832872889 (203h11m51.26395302s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XHNZDY7xKZ2bss1rSdjbGP3ugSwGRwRzBSWHfXgdKKz794wMq): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832892156 (203h11m51.263972287s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832974564 (203h11m51.264054695s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa): protocol: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection Aug 17 04:37:56.833025241 (203h11m51.264105372s) [seg_pos={0 0}, upload=211361]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:37:56.833116014 (203h11m51.264196135s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:37:56.833120332 (203h11m51.264200463s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:37:58.261163702 (203h11m52.692243834s) [upload=211361]: Done uploading segments... Aug 17 04:37:58.261181997 (203h11m52.692262118s) [upload=211361]: Upload finished. err: <nil> Duration: 5.777195578s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................XXXX................................] duration: 5.834242622s FAIL (2025-08-17 04:46:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.83426815s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 6.356424556s FAIL (2025-08-17 04:46:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.356445466s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.....X........] duration: 1.788941732s FAIL (2025-08-17 04:46:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.788958554s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................XXXX...............................] duration: 5.980505632s FAIL (2025-08-17 04:46:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.980537103s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X.....X.......] duration: 2.048068686s FAIL (2025-08-17 04:46:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 2.048092552s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................XXXX...............................] duration: 6.55944718s FAIL (2025-08-17 04:46:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.559470837s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................XXX.................................] duration: 5.754290602s FAIL (2025-08-17 04:46:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.754321081s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [..................X..........XXXX.X.X.......X...X...........] duration: 5.704678458s FAIL (2025-08-17 04:46:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 17 04:37:54.182428668 (203h11m48.613508799s) [upload=211361]: Starting upload Aug 17 04:37:54.182474656 (203h11m48.613554777s) [upload=211361]: Done starting upload Aug 17 04:37:54.182665000 (203h11m48.613745131s) [upload=211361]: Uploading segments... Aug 17 04:37:54.182728421 (203h11m48.613808552s) [upload=211361]: Got next segment. Inline: false Aug 17 04:37:54.182758048 (203h11m48.613838179s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:37:54.182762306 (203h11m48.613842437s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:37:54.737160222 (203h11m49.168240353s) [seg_pos={0 0}, upload=211361]: Begin upload segment... Aug 17 04:37:54.737203494 (203h11m49.168283625s) [seg_pos={0 0}, upload=211361]: Done begin upload segment. Aug 17 04:37:54.737224945 (203h11m49.168305076s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737311591 (203h11m49.168391722s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737638917 (203h11m49.168719048s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737939432 (203h11m49.169019563s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737952067 (203h11m49.169032198s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738229999 (203h11m49.16931013s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738324038 (203h11m49.169404169s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738511367 (203h11m49.169591488s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738655813 (203h11m49.169735954s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738747769 (203h11m49.1698279s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738971367 (203h11m49.170051498s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738975705 (203h11m49.170055836s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739194995 (203h11m49.170275126s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739278324 (203h11m49.170358445s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739421147 (203h11m49.170501278s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739561055 (203h11m49.170641186s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739643824 (203h11m49.170723955s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739852683 (203h11m49.170932814s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:55.861243533 (203h11m50.292323664s) [upload=211361]: Next returned nil segment Aug 17 04:37:55.861256768 (203h11m50.292336899s) [upload=211361]: Waiting for error group managing segments... Aug 17 04:37:56.165335502 (203h11m50.596415633s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.263548583 (203h11m50.694628715s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.569392606 (203h11m51.000472737s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.620901052 (203h11m51.051981183s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.639713969 (203h11m51.0707941s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.649068307 (203h11m51.080148438s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.701677831 (203h11m51.132757962s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.719713991 (203h11m51.150794132s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.730632311 (203h11m51.161712432s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.761149974 (203h11m51.192230105s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.775738719 (203h11m51.20681885s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.779316274 (203h11m51.210396405s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.802147800 (203h11m51.233227961s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832614886 (203h11m51.263695017s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832646346 (203h11m51.263726467s) [seg_pos={0 0}, upload=211361]: Segment reached optimal threshold of 14 pieces. Aug 17 04:37:56.832765443 (203h11m51.263845574s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832872889 (203h11m51.26395302s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XHNZDY7xKZ2bss1rSdjbGP3ugSwGRwRzBSWHfXgdKKz794wMq): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832892156 (203h11m51.263972287s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832974564 (203h11m51.264054695s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa): protocol: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection Aug 17 04:37:56.833025241 (203h11m51.264105372s) [seg_pos={0 0}, upload=211361]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:37:56.833116014 (203h11m51.264196135s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:37:56.833120332 (203h11m51.264200463s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:37:58.261163702 (203h11m52.692243834s) [upload=211361]: Done uploading segments... Aug 17 04:37:58.261181997 (203h11m52.692262118s) [upload=211361]: Upload finished. err: <nil> Duration: 5.704710529s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................XXXX..............................] duration: 5.862394658s FAIL (2025-08-17 04:45:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.862439925s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................XXXX..............................] duration: 6.103545186s FAIL (2025-08-17 04:45:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.103565837s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................XXX................................] duration: 5.88276289s FAIL (2025-08-17 04:45:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.882790484s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 6.530718389s FAIL (2025-08-17 04:45:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.53073991s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................X..........XXXX.X.X.......X...X..........] duration: 5.552087906s FAIL (2025-08-17 04:45:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 17 04:37:54.182428668 (203h11m48.613508799s) [upload=211361]: Starting upload Aug 17 04:37:54.182474656 (203h11m48.613554777s) [upload=211361]: Done starting upload Aug 17 04:37:54.182665000 (203h11m48.613745131s) [upload=211361]: Uploading segments... Aug 17 04:37:54.182728421 (203h11m48.613808552s) [upload=211361]: Got next segment. Inline: false Aug 17 04:37:54.182758048 (203h11m48.613838179s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:37:54.182762306 (203h11m48.613842437s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:37:54.737160222 (203h11m49.168240353s) [seg_pos={0 0}, upload=211361]: Begin upload segment... Aug 17 04:37:54.737203494 (203h11m49.168283625s) [seg_pos={0 0}, upload=211361]: Done begin upload segment. Aug 17 04:37:54.737224945 (203h11m49.168305076s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737311591 (203h11m49.168391722s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737638917 (203h11m49.168719048s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737939432 (203h11m49.169019563s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737952067 (203h11m49.169032198s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738229999 (203h11m49.16931013s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738324038 (203h11m49.169404169s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738511367 (203h11m49.169591488s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738655813 (203h11m49.169735954s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738747769 (203h11m49.1698279s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738971367 (203h11m49.170051498s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738975705 (203h11m49.170055836s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739194995 (203h11m49.170275126s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739278324 (203h11m49.170358445s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739421147 (203h11m49.170501278s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739561055 (203h11m49.170641186s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739643824 (203h11m49.170723955s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739852683 (203h11m49.170932814s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:55.861243533 (203h11m50.292323664s) [upload=211361]: Next returned nil segment Aug 17 04:37:55.861256768 (203h11m50.292336899s) [upload=211361]: Waiting for error group managing segments... Aug 17 04:37:56.165335502 (203h11m50.596415633s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.263548583 (203h11m50.694628715s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.569392606 (203h11m51.000472737s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.620901052 (203h11m51.051981183s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.639713969 (203h11m51.0707941s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.649068307 (203h11m51.080148438s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.701677831 (203h11m51.132757962s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.719713991 (203h11m51.150794132s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.730632311 (203h11m51.161712432s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.761149974 (203h11m51.192230105s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.775738719 (203h11m51.20681885s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.779316274 (203h11m51.210396405s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.802147800 (203h11m51.233227961s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832614886 (203h11m51.263695017s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832646346 (203h11m51.263726467s) [seg_pos={0 0}, upload=211361]: Segment reached optimal threshold of 14 pieces. Aug 17 04:37:56.832765443 (203h11m51.263845574s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832872889 (203h11m51.26395302s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XHNZDY7xKZ2bss1rSdjbGP3ugSwGRwRzBSWHfXgdKKz794wMq): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832892156 (203h11m51.263972287s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832974564 (203h11m51.264054695s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa): protocol: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection Aug 17 04:37:56.833025241 (203h11m51.264105372s) [seg_pos={0 0}, upload=211361]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:37:56.833116014 (203h11m51.264196135s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:37:56.833120332 (203h11m51.264200463s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:37:58.261163702 (203h11m52.692243834s) [upload=211361]: Done uploading segments... Aug 17 04:37:58.261181997 (203h11m52.692262118s) [upload=211361]: Upload finished. err: <nil> Duration: 5.552129637s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................XXXX.............................] duration: 6.229643152s FAIL (2025-08-17 04:45:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.229667208s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................XXXX.............................] duration: 6.320001635s FAIL (2025-08-17 04:45:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.320025702s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X.....X......] duration: 1.86839062s FAIL (2025-08-17 04:45:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.868417681s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................XXX...............................] duration: 5.978535905s FAIL (2025-08-17 04:45:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.978559109s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [....................X..........XXXX.X.X.......X...X.........] duration: 5.872567333s FAIL (2025-08-17 04:45:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 17 04:37:54.182428668 (203h11m48.613508799s) [upload=211361]: Starting upload Aug 17 04:37:54.182474656 (203h11m48.613554777s) [upload=211361]: Done starting upload Aug 17 04:37:54.182665000 (203h11m48.613745131s) [upload=211361]: Uploading segments... Aug 17 04:37:54.182728421 (203h11m48.613808552s) [upload=211361]: Got next segment. Inline: false Aug 17 04:37:54.182758048 (203h11m48.613838179s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:37:54.182762306 (203h11m48.613842437s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:37:54.737160222 (203h11m49.168240353s) [seg_pos={0 0}, upload=211361]: Begin upload segment... Aug 17 04:37:54.737203494 (203h11m49.168283625s) [seg_pos={0 0}, upload=211361]: Done begin upload segment. Aug 17 04:37:54.737224945 (203h11m49.168305076s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737311591 (203h11m49.168391722s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737638917 (203h11m49.168719048s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737939432 (203h11m49.169019563s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737952067 (203h11m49.169032198s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738229999 (203h11m49.16931013s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738324038 (203h11m49.169404169s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738511367 (203h11m49.169591488s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738655813 (203h11m49.169735954s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738747769 (203h11m49.1698279s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738971367 (203h11m49.170051498s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738975705 (203h11m49.170055836s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739194995 (203h11m49.170275126s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739278324 (203h11m49.170358445s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739421147 (203h11m49.170501278s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739561055 (203h11m49.170641186s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739643824 (203h11m49.170723955s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739852683 (203h11m49.170932814s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:55.861243533 (203h11m50.292323664s) [upload=211361]: Next returned nil segment Aug 17 04:37:55.861256768 (203h11m50.292336899s) [upload=211361]: Waiting for error group managing segments... Aug 17 04:37:56.165335502 (203h11m50.596415633s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.263548583 (203h11m50.694628715s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.569392606 (203h11m51.000472737s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.620901052 (203h11m51.051981183s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.639713969 (203h11m51.0707941s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.649068307 (203h11m51.080148438s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.701677831 (203h11m51.132757962s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.719713991 (203h11m51.150794132s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.730632311 (203h11m51.161712432s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.761149974 (203h11m51.192230105s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.775738719 (203h11m51.20681885s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.779316274 (203h11m51.210396405s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.802147800 (203h11m51.233227961s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832614886 (203h11m51.263695017s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832646346 (203h11m51.263726467s) [seg_pos={0 0}, upload=211361]: Segment reached optimal threshold of 14 pieces. Aug 17 04:37:56.832765443 (203h11m51.263845574s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832872889 (203h11m51.26395302s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XHNZDY7xKZ2bss1rSdjbGP3ugSwGRwRzBSWHfXgdKKz794wMq): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832892156 (203h11m51.263972287s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832974564 (203h11m51.264054695s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa): protocol: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection Aug 17 04:37:56.833025241 (203h11m51.264105372s) [seg_pos={0 0}, upload=211361]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:37:56.833116014 (203h11m51.264196135s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:37:56.833120332 (203h11m51.264200463s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:37:58.261163702 (203h11m52.692243834s) [upload=211361]: Done uploading segments... Aug 17 04:37:58.261181997 (203h11m52.692262118s) [upload=211361]: Upload finished. err: <nil> Duration: 5.87259134s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................XXXX............................] duration: 5.781148042s FAIL (2025-08-17 04:44:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.781172529s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X...............................................X.....X.....] duration: 1.904539227s FAIL (2025-08-17 04:44:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.904561689s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................XXX..............................] duration: 5.588393224s FAIL (2025-08-17 04:44:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.5884169s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 6.775731306s FAIL (2025-08-17 04:44:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.775755202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....................X..........XXXX.X.X.......X...X........] duration: 6.014362614s FAIL (2025-08-17 04:44:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 17 04:37:54.182428668 (203h11m48.613508799s) [upload=211361]: Starting upload Aug 17 04:37:54.182474656 (203h11m48.613554777s) [upload=211361]: Done starting upload Aug 17 04:37:54.182665000 (203h11m48.613745131s) [upload=211361]: Uploading segments... Aug 17 04:37:54.182728421 (203h11m48.613808552s) [upload=211361]: Got next segment. Inline: false Aug 17 04:37:54.182758048 (203h11m48.613838179s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:37:54.182762306 (203h11m48.613842437s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:37:54.737160222 (203h11m49.168240353s) [seg_pos={0 0}, upload=211361]: Begin upload segment... Aug 17 04:37:54.737203494 (203h11m49.168283625s) [seg_pos={0 0}, upload=211361]: Done begin upload segment. Aug 17 04:37:54.737224945 (203h11m49.168305076s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737311591 (203h11m49.168391722s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737638917 (203h11m49.168719048s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737939432 (203h11m49.169019563s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737952067 (203h11m49.169032198s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738229999 (203h11m49.16931013s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738324038 (203h11m49.169404169s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738511367 (203h11m49.169591488s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738655813 (203h11m49.169735954s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738747769 (203h11m49.1698279s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738971367 (203h11m49.170051498s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738975705 (203h11m49.170055836s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739194995 (203h11m49.170275126s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739278324 (203h11m49.170358445s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739421147 (203h11m49.170501278s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739561055 (203h11m49.170641186s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739643824 (203h11m49.170723955s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739852683 (203h11m49.170932814s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:55.861243533 (203h11m50.292323664s) [upload=211361]: Next returned nil segment Aug 17 04:37:55.861256768 (203h11m50.292336899s) [upload=211361]: Waiting for error group managing segments... Aug 17 04:37:56.165335502 (203h11m50.596415633s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.263548583 (203h11m50.694628715s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.569392606 (203h11m51.000472737s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.620901052 (203h11m51.051981183s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.639713969 (203h11m51.0707941s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.649068307 (203h11m51.080148438s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.701677831 (203h11m51.132757962s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.719713991 (203h11m51.150794132s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.730632311 (203h11m51.161712432s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.761149974 (203h11m51.192230105s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.775738719 (203h11m51.20681885s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.779316274 (203h11m51.210396405s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.802147800 (203h11m51.233227961s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832614886 (203h11m51.263695017s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832646346 (203h11m51.263726467s) [seg_pos={0 0}, upload=211361]: Segment reached optimal threshold of 14 pieces. Aug 17 04:37:56.832765443 (203h11m51.263845574s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832872889 (203h11m51.26395302s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XHNZDY7xKZ2bss1rSdjbGP3ugSwGRwRzBSWHfXgdKKz794wMq): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832892156 (203h11m51.263972287s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832974564 (203h11m51.264054695s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa): protocol: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection Aug 17 04:37:56.833025241 (203h11m51.264105372s) [seg_pos={0 0}, upload=211361]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:37:56.833116014 (203h11m51.264196135s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:37:56.833120332 (203h11m51.264200463s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:37:58.261163702 (203h11m52.692243834s) [upload=211361]: Done uploading segments... Aug 17 04:37:58.261181997 (203h11m52.692262118s) [upload=211361]: Upload finished. err: <nil> Duration: 6.014393604s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................XXXX............................] duration: 6.232688028s FAIL (2025-08-17 04:44:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.232715701s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [......................X..........XXXX.X.X.......X...X.......] duration: 5.613804135s FAIL (2025-08-17 04:43:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 17 04:37:54.182428668 (203h11m48.613508799s) [upload=211361]: Starting upload Aug 17 04:37:54.182474656 (203h11m48.613554777s) [upload=211361]: Done starting upload Aug 17 04:37:54.182665000 (203h11m48.613745131s) [upload=211361]: Uploading segments... Aug 17 04:37:54.182728421 (203h11m48.613808552s) [upload=211361]: Got next segment. Inline: false Aug 17 04:37:54.182758048 (203h11m48.613838179s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:37:54.182762306 (203h11m48.613842437s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:37:54.737160222 (203h11m49.168240353s) [seg_pos={0 0}, upload=211361]: Begin upload segment... Aug 17 04:37:54.737203494 (203h11m49.168283625s) [seg_pos={0 0}, upload=211361]: Done begin upload segment. Aug 17 04:37:54.737224945 (203h11m49.168305076s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737311591 (203h11m49.168391722s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737638917 (203h11m49.168719048s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737939432 (203h11m49.169019563s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737952067 (203h11m49.169032198s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738229999 (203h11m49.16931013s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738324038 (203h11m49.169404169s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738511367 (203h11m49.169591488s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738655813 (203h11m49.169735954s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738747769 (203h11m49.1698279s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738971367 (203h11m49.170051498s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738975705 (203h11m49.170055836s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739194995 (203h11m49.170275126s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739278324 (203h11m49.170358445s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739421147 (203h11m49.170501278s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739561055 (203h11m49.170641186s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739643824 (203h11m49.170723955s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739852683 (203h11m49.170932814s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:55.861243533 (203h11m50.292323664s) [upload=211361]: Next returned nil segment Aug 17 04:37:55.861256768 (203h11m50.292336899s) [upload=211361]: Waiting for error group managing segments... Aug 17 04:37:56.165335502 (203h11m50.596415633s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.263548583 (203h11m50.694628715s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.569392606 (203h11m51.000472737s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.620901052 (203h11m51.051981183s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.639713969 (203h11m51.0707941s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.649068307 (203h11m51.080148438s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.701677831 (203h11m51.132757962s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.719713991 (203h11m51.150794132s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.730632311 (203h11m51.161712432s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.761149974 (203h11m51.192230105s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.775738719 (203h11m51.20681885s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.779316274 (203h11m51.210396405s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.802147800 (203h11m51.233227961s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832614886 (203h11m51.263695017s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832646346 (203h11m51.263726467s) [seg_pos={0 0}, upload=211361]: Segment reached optimal threshold of 14 pieces. Aug 17 04:37:56.832765443 (203h11m51.263845574s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832872889 (203h11m51.26395302s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XHNZDY7xKZ2bss1rSdjbGP3ugSwGRwRzBSWHfXgdKKz794wMq): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832892156 (203h11m51.263972287s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832974564 (203h11m51.264054695s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa): protocol: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection Aug 17 04:37:56.833025241 (203h11m51.264105372s) [seg_pos={0 0}, upload=211361]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:37:56.833116014 (203h11m51.264196135s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:37:56.833120332 (203h11m51.264200463s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:37:58.261163702 (203h11m52.692243834s) [upload=211361]: Done uploading segments... Aug 17 04:37:58.261181997 (203h11m52.692262118s) [upload=211361]: Upload finished. err: <nil> Duration: 5.613830144s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................XXXX...........................] duration: 6.012168845s FAIL (2025-08-17 04:43:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.012193954s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X...............................................X.....X....] duration: 1.805145426s FAIL (2025-08-17 04:43:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.805175173s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 6.397284239s FAIL (2025-08-17 04:43:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.3973061s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................XXX.............................] duration: 5.748954364s FAIL (2025-08-17 04:43:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.748972117s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................XXXX...........................] duration: 6.244546798s FAIL (2025-08-17 04:43:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.244567629s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [X......................X..........XXXX.X.X.......X...X......] duration: 5.992855312s FAIL (2025-08-17 04:43:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 17 04:37:54.182428668 (203h11m48.613508799s) [upload=211361]: Starting upload Aug 17 04:37:54.182474656 (203h11m48.613554777s) [upload=211361]: Done starting upload Aug 17 04:37:54.182665000 (203h11m48.613745131s) [upload=211361]: Uploading segments... Aug 17 04:37:54.182728421 (203h11m48.613808552s) [upload=211361]: Got next segment. Inline: false Aug 17 04:37:54.182758048 (203h11m48.613838179s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:37:54.182762306 (203h11m48.613842437s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:37:54.737160222 (203h11m49.168240353s) [seg_pos={0 0}, upload=211361]: Begin upload segment... Aug 17 04:37:54.737203494 (203h11m49.168283625s) [seg_pos={0 0}, upload=211361]: Done begin upload segment. Aug 17 04:37:54.737224945 (203h11m49.168305076s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737311591 (203h11m49.168391722s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737638917 (203h11m49.168719048s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737939432 (203h11m49.169019563s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737952067 (203h11m49.169032198s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738229999 (203h11m49.16931013s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738324038 (203h11m49.169404169s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738511367 (203h11m49.169591488s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738655813 (203h11m49.169735954s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738747769 (203h11m49.1698279s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738971367 (203h11m49.170051498s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738975705 (203h11m49.170055836s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739194995 (203h11m49.170275126s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739278324 (203h11m49.170358445s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739421147 (203h11m49.170501278s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739561055 (203h11m49.170641186s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739643824 (203h11m49.170723955s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739852683 (203h11m49.170932814s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:55.861243533 (203h11m50.292323664s) [upload=211361]: Next returned nil segment Aug 17 04:37:55.861256768 (203h11m50.292336899s) [upload=211361]: Waiting for error group managing segments... Aug 17 04:37:56.165335502 (203h11m50.596415633s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.263548583 (203h11m50.694628715s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.569392606 (203h11m51.000472737s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.620901052 (203h11m51.051981183s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.639713969 (203h11m51.0707941s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.649068307 (203h11m51.080148438s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.701677831 (203h11m51.132757962s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.719713991 (203h11m51.150794132s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.730632311 (203h11m51.161712432s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.761149974 (203h11m51.192230105s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.775738719 (203h11m51.20681885s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.779316274 (203h11m51.210396405s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.802147800 (203h11m51.233227961s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832614886 (203h11m51.263695017s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832646346 (203h11m51.263726467s) [seg_pos={0 0}, upload=211361]: Segment reached optimal threshold of 14 pieces. Aug 17 04:37:56.832765443 (203h11m51.263845574s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832872889 (203h11m51.26395302s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XHNZDY7xKZ2bss1rSdjbGP3ugSwGRwRzBSWHfXgdKKz794wMq): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832892156 (203h11m51.263972287s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832974564 (203h11m51.264054695s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa): protocol: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection Aug 17 04:37:56.833025241 (203h11m51.264105372s) [seg_pos={0 0}, upload=211361]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:37:56.833116014 (203h11m51.264196135s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:37:56.833120332 (203h11m51.264200463s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:37:58.261163702 (203h11m52.692243834s) [upload=211361]: Done uploading segments... Aug 17 04:37:58.261181997 (203h11m52.692262118s) [upload=211361]: Upload finished. err: <nil> Duration: 5.992881854s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................XXXX..........................] duration: 5.971492949s FAIL (2025-08-17 04:43:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.971525711s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X...............................................X.....X...] duration: 2.162966993s FAIL (2025-08-17 04:43:03 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 2.162989235s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 6.51506536s FAIL (2025-08-17 04:43:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.515080538s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [.X......................X..........XXXX.X.X.......X...X.....] duration: 5.931076695s FAIL (2025-08-17 04:42:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 17 04:37:54.182428668 (203h11m48.613508799s) [upload=211361]: Starting upload Aug 17 04:37:54.182474656 (203h11m48.613554777s) [upload=211361]: Done starting upload Aug 17 04:37:54.182665000 (203h11m48.613745131s) [upload=211361]: Uploading segments... Aug 17 04:37:54.182728421 (203h11m48.613808552s) [upload=211361]: Got next segment. Inline: false Aug 17 04:37:54.182758048 (203h11m48.613838179s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:37:54.182762306 (203h11m48.613842437s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:37:54.737160222 (203h11m49.168240353s) [seg_pos={0 0}, upload=211361]: Begin upload segment... Aug 17 04:37:54.737203494 (203h11m49.168283625s) [seg_pos={0 0}, upload=211361]: Done begin upload segment. Aug 17 04:37:54.737224945 (203h11m49.168305076s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737311591 (203h11m49.168391722s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737638917 (203h11m49.168719048s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737939432 (203h11m49.169019563s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.737952067 (203h11m49.169032198s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738229999 (203h11m49.16931013s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738324038 (203h11m49.169404169s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738511367 (203h11m49.169591488s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738655813 (203h11m49.169735954s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738747769 (203h11m49.1698279s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738971367 (203h11m49.170051498s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.738975705 (203h11m49.170055836s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739194995 (203h11m49.170275126s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739278324 (203h11m49.170358445s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739421147 (203h11m49.170501278s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739561055 (203h11m49.170641186s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739643824 (203h11m49.170723955s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:54.739852683 (203h11m49.170932814s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Uploading piece... Aug 17 04:37:55.861243533 (203h11m50.292323664s) [upload=211361]: Next returned nil segment Aug 17 04:37:55.861256768 (203h11m50.292336899s) [upload=211361]: Waiting for error group managing segments... Aug 17 04:37:56.165335502 (203h11m50.596415633s) [noise=true , address=66.23.193.175:4562 , piece_id=2RZ2A5KBLZBLEG6UECY2GTNF5THR3XJZMS34ULG5SQNWN4O3XNMA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.263548583 (203h11m50.694628715s) [noise=true , address=66.23.193.175:4563 , piece_id=I65W2GP5RTRZUIT7Y6NSEAL7XC726PP3BYMCWS6UMZEOWAP5CP2A, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.569392606 (203h11m51.000472737s) [noise=true , address=79.127.220.101:4068 , piece_id=6MFJ22JL437NTOH25E3LNH3Q2LQNKKMUT5Q43F2XNIB7346LGFCQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.620901052 (203h11m51.051981183s) [noise=true , address=77.90.31.12:4601 , piece_id=JC5VIGLKCWTCDHQYWEL6QSRZW3MJZFRFQKRJH54OK7LPUQH7WQTQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.639713969 (203h11m51.0707941s) [noise=true , address=77.90.31.16:4761 , piece_id=44YO7GYP4RNRWHPP7LP2C4FJBGMWOLEDEC2NN7TAWU3YFHAST3XQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.649068307 (203h11m51.080148438s) [noise=true , address=104.36.82.4:4833 , piece_id=AMRE5SIA7EJLAP55GFYILJYZJL5624E534TPWFUEQILHG7CCFDXQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.701677831 (203h11m51.132757962s) [noise=true , address=104.36.82.3:4800 , piece_id=EKC43HOMYNG77PDESU3QV5OL44HWYQRAN5RUNIB7IMC6NVXQWFVA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.719713991 (203h11m51.150794132s) [noise=true , address=104.36.82.23:4468 , piece_id=NC2BFOASSHA4ZG46LZONWF23DNN3ZZBA6SFO3Q44KD4TCOH4AIQQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.730632311 (203h11m51.161712432s) [noise=true , address=104.36.82.39:4747 , piece_id=SF2P47ED5FEOCMJ6R6AXP2NTI5LYYALSDLHINNLDVMUMZIE4YLHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.761149974 (203h11m51.192230105s) [noise=true , address=104.36.82.60:4360 , piece_id=W4OJU6NRV2HKPC7JDTISAPVT77KIC3JHVJR5KHYY225FLH4PZBBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.775738719 (203h11m51.20681885s) [noise=true , address=77.90.31.18:4830 , piece_id=QNQF4JAEHOFWRTL6XWKL42XSKA2DMO6Z4CZO5NF337NNSHCMYYBA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.779316274 (203h11m51.210396405s) [noise=true , address=77.90.31.13:4634 , piece_id=ZXX5E54TSYUG557OGOCBQMBKS35TUG6OPDOIG3TTS3OYGTRA24XA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.802147800 (203h11m51.233227961s) [noise=true , address=104.36.82.58:4286 , piece_id=JDBFFYOZPYJ5FXJQIVEBB5IWSBGT6LCBUZF5O25OKWFRXROPHS3Q, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832614886 (203h11m51.263695017s) [noise=true , address=104.36.82.44:4876 , piece_id=H53T7WMGX4CDVQNCYOG6NOYFVRHD5Z35LNF37FVKSWYIVJS2EFZA, seg_pos={0 0}, upload=211361]: Done uploading piece. err: <nil> Aug 17 04:37:56.832646346 (203h11m51.263726467s) [seg_pos={0 0}, upload=211361]: Segment reached optimal threshold of 14 pieces. Aug 17 04:37:56.832765443 (203h11m51.263845574s) [noise=true , address=77.90.31.10:4532 , piece_id=5IBCLJNM6LUBCIOPSAERY5HAXV66CR4OKI45RBGOHALTRFYGLMWQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832872889 (203h11m51.26395302s) [noise=true , address=104.36.82.36:4648 , piece_id=GTHXHUS6GR26G6UHHMB35MCSEI7I5UJ6K65O76HRJI4WVPIHD7RQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XHNZDY7xKZ2bss1rSdjbGP3ugSwGRwRzBSWHfXgdKKz794wMq): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832892156 (203h11m51.263972287s) [noise=true , address=77.90.31.19:4863 , piece_id=PYERWDVDOTOW2SVDUO63ODTEWMWUGH5OMV2PE7RJTIDVFTF2ZRVQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Aug 17 04:37:56.832974564 (203h11m51.264054695s) [noise=true , address=104.36.82.5:4844 , piece_id=PULFFEXZLSMOYEFGRK6L2POF7ERJQT75J23JT3CTXKZ3LMARQZHQ, seg_pos={0 0}, upload=211361]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa): protocol: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59826->104.36.82.5:4844: use of closed network connection Aug 17 04:37:56.833025241 (203h11m51.264105372s) [seg_pos={0 0}, upload=211361]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:37:56.833116014 (203h11m51.264196135s) [upload=211361]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:37:56.833120332 (203h11m51.264200463s) [upload=211361]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:37:58.261163702 (203h11m52.692243834s) [upload=211361]: Done uploading segments... Aug 17 04:37:58.261181997 (203h11m52.692262118s) [upload=211361]: Upload finished. err: <nil> Duration: 5.931127814s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................XXXX..........................] duration: 6.228288949s FAIL (2025-08-17 04:42:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.228319098s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 6.991773523s FAIL (2025-08-17 04:42:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.991791719s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X...............................................X.....X..] duration: 1.886894184s FAIL (2025-08-17 04:42:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.886912478s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................XXX............................] duration: 5.87043373s FAIL (2025-08-17 04:42:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.870467975s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....X...............................................X.....X.] duration: 1.854283648s FAIL (2025-08-17 04:42:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.854306831s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................XXXX.........................] duration: 6.228677968s FAIL (2025-08-17 04:42:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.22870083s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................XXXX.........................] duration: 5.738167874s FAIL (2025-08-17 04:42:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.738190908s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 6.306776443s FAIL (2025-08-17 04:41:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.306825617s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)