Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 7.241107438s FAIL (2025-08-10 23:07:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.24112957s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 10.138076811s FAIL (2025-08-10 23:07:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 10.138095767s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [..........X.......X.X.....X.X...........XX...............X..] duration: 10.437923247s FAIL (2025-08-10 23:07:03 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 10 23:05:03.554619361 (53h38m57.985699492s) [upload=55940]: Starting upload Aug 10 23:05:03.554665109 (53h38m57.98574524s) [upload=55940]: Done starting upload Aug 10 23:05:03.554811649 (53h38m57.98589179s) [upload=55940]: Uploading segments... Aug 10 23:05:03.554899368 (53h38m57.985979499s) [upload=55940]: Got next segment. Inline: false Aug 10 23:05:03.554964262 (53h38m57.986044393s) [upload=55940]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:05:03.554971365 (53h38m57.986051496s) [upload=55940]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:05:04.007203761 (53h38m58.438283892s) [seg_pos={0 0}, upload=55940]: Begin upload segment... Aug 10 23:05:04.007274326 (53h38m58.438354457s) [seg_pos={0 0}, upload=55940]: Done begin upload segment. Aug 10 23:05:04.007477475 (53h38m58.438557606s) [noise=true , address=104.36.82.12:4182 , piece_id=KXVG5327X5P3GMHMJW5UGL7PWMI2ZRQF6BXDWZXAUJ2AP7DKKK3A, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.007625788 (53h38m58.438705929s) [noise=true , address=104.36.82.4:4836 , piece_id=PAEI5LLGPCMAYKBKRTD7QXQGC6KDP4E6QCXLJYKBTMWZEM5IKXIA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.007960198 (53h38m58.439040329s) [noise=true , address=104.36.82.61:4383 , piece_id=6ZNA5JKIQCDP4RH42V7AAADEYYDJKUARDHJJWBLHEHBK3POXRWQQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.008859278 (53h38m58.43993941s) [noise=true , address=104.36.82.37:4663 , piece_id=RMC6ORT4SG3A7B3QB5L72ACQ3V5YY734BKIGRIXCWC5ADO6LBH3A, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.009284422 (53h38m58.440364584s) [noise=true , address=104.36.82.23:4457 , piece_id=OTRABU246BYQQMZKGWVNL7LFI2KLHQ3IGY6RR6CXIZL4M6TQDCVA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.009684147 (53h38m58.440764278s) [noise=true , address=45.91.116.9:4287 , piece_id=ZJGRIG6XWTNVG3GYBP5N4WJQLXYCPJW6JVLRMYA4CYX5SO7XKWYA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.010316126 (53h38m58.441396257s) [noise=true , address=104.36.82.58:4274 , piece_id=A7QKWC3YIMDBTIMOW5AKLE72I533AGAAOV66XEQEBANUTL72QBHQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.010614587 (53h38m58.441694718s) [noise=true , address=104.36.82.34:4570 , piece_id=J5CPBQHYKFZHZFZKPDLYG4G2P64P5MFTYYJVEK4NRN4MYBV2DEVA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.011014984 (53h38m58.442095115s) [noise=true , address=104.36.82.19:4407 , piece_id=M7VNWXU3KAKR6YV66VYBNRYICGT7PAIVMGINKIJJ6PNV2FB3RXSA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.011474642 (53h38m58.442554773s) [noise=true , address=104.36.82.35:4583 , piece_id=AOQELQSBY44FQTFEOSAE4ZFRF7P55OKOD7DHICQXW6L57RQRYYGQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.011921518 (53h38m58.443001649s) [noise=true , address=104.36.82.14:4247 , piece_id=QEHL3O75QFJ23FX3AZJSO2CBPFFYUELN3LLPZHFFQMZH4IPP2C2A, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.012318427 (53h38m58.443398558s) [noise=true , address=104.36.82.56:4231 , piece_id=RD4JOJGFE2CQZKDWIBBT56RRV2VDFDTQU4DLNZRMQTL6VOI422JA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.012784880 (53h38m58.443865011s) [noise=true , address=64.49.13.26:4947 , piece_id=CYBTXVHRWDPO54OFXDGQQAZ26YQS4DR7VT34POHNBRI3YFBYXJ7Q, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.014247087 (53h38m58.445327218s) [noise=true , address=104.36.82.38:4697 , piece_id=WX3FOUYDSWKPZW6D6WZFW4RXU2YIESB5OJTBPTH55EM5QB2JIMBQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.015466491 (53h38m58.446546622s) [noise=true , address=104.36.82.46:4499 , piece_id=2HPG6CKLRYXLYI6RQQWU2PHTESFLTH6ZVSKSILTX3OJLYVNTWSMQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.016638493 (53h38m58.447718624s) [noise=true , address=104.36.82.41:4777 , piece_id=UF5K4GTU2SW5LCHZ5APEDHJMSDMP64FA73LANW2VUE2GTCIOZMBA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.017984358 (53h38m58.449064489s) [noise=true , address=104.36.82.15:4269 , piece_id=JHKDOTBRLLH6YPR6VKOFVLPA3Q6YSMGUN4IF66ORDHS2W66FFYRQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.019254679 (53h38m58.45033482s) [noise=true , address=104.36.82.7:4936 , piece_id=2GTIP5HA7ZN4X3ACCD43TSVJT74NNIMWLPBVDYI7UIXQTBDKLD6Q, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:05.149856335 (53h38m59.580936466s) [upload=55940]: Next returned nil segment Aug 10 23:05:05.149871423 (53h38m59.580951554s) [upload=55940]: Waiting for error group managing segments... Aug 10 23:05:05.439410926 (53h38m59.870491057s) [noise=true , address=64.49.13.26:4947 , piece_id=CYBTXVHRWDPO54OFXDGQQAZ26YQS4DR7VT34POHNBRI3YFBYXJ7Q, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.051323200 (53h39m0.482403331s) [noise=true , address=104.36.82.56:4231 , piece_id=RD4JOJGFE2CQZKDWIBBT56RRV2VDFDTQU4DLNZRMQTL6VOI422JA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.075755667 (53h39m0.506835788s) [noise=true , address=104.36.82.7:4936 , piece_id=2GTIP5HA7ZN4X3ACCD43TSVJT74NNIMWLPBVDYI7UIXQTBDKLD6Q, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.097055631 (53h39m0.528135762s) [noise=true , address=104.36.82.46:4499 , piece_id=2HPG6CKLRYXLYI6RQQWU2PHTESFLTH6ZVSKSILTX3OJLYVNTWSMQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.098401586 (53h39m0.529481707s) [noise=true , address=104.36.82.34:4570 , piece_id=J5CPBQHYKFZHZFZKPDLYG4G2P64P5MFTYYJVEK4NRN4MYBV2DEVA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.125020757 (53h39m0.556100888s) [noise=true , address=104.36.82.58:4274 , piece_id=A7QKWC3YIMDBTIMOW5AKLE72I533AGAAOV66XEQEBANUTL72QBHQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.147088209 (53h39m0.57816838s) [noise=true , address=104.36.82.19:4407 , piece_id=M7VNWXU3KAKR6YV66VYBNRYICGT7PAIVMGINKIJJ6PNV2FB3RXSA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.161291916 (53h39m0.592372047s) [noise=true , address=104.36.82.23:4457 , piece_id=OTRABU246BYQQMZKGWVNL7LFI2KLHQ3IGY6RR6CXIZL4M6TQDCVA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.186986768 (53h39m0.618066889s) [noise=true , address=104.36.82.12:4182 , piece_id=KXVG5327X5P3GMHMJW5UGL7PWMI2ZRQF6BXDWZXAUJ2AP7DKKK3A, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.324393872 (53h39m0.755474003s) [noise=true , address=45.91.116.9:4287 , piece_id=ZJGRIG6XWTNVG3GYBP5N4WJQLXYCPJW6JVLRMYA4CYX5SO7XKWYA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.502418269 (53h39m0.9334984s) [noise=true , address=104.36.82.15:4269 , piece_id=JHKDOTBRLLH6YPR6VKOFVLPA3Q6YSMGUN4IF66ORDHS2W66FFYRQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.546654377 (53h39m0.977734509s) [noise=true , address=104.36.82.37:4663 , piece_id=RMC6ORT4SG3A7B3QB5L72ACQ3V5YY734BKIGRIXCWC5ADO6LBH3A, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.827166137 (53h39m1.258246268s) [noise=true , address=104.36.82.4:4836 , piece_id=PAEI5LLGPCMAYKBKRTD7QXQGC6KDP4E6QCXLJYKBTMWZEM5IKXIA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:07.698669291 (53h39m2.129749422s) [noise=true , address=104.36.82.38:4697 , piece_id=WX3FOUYDSWKPZW6D6WZFW4RXU2YIESB5OJTBPTH55EM5QB2JIMBQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:07.698700771 (53h39m2.129780892s) [seg_pos={0 0}, upload=55940]: Segment reached optimal threshold of 14 pieces. Aug 10 23:05:07.698835169 (53h39m2.1299153s) [noise=true , address=104.36.82.41:4777 , piece_id=UF5K4GTU2SW5LCHZ5APEDHJMSDMP64FA73LANW2VUE2GTCIOZMBA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Pvm1SftEF3tnCEBqUn1ZmyVcLDBkR6pyegEb7mtzjdQiiqYqt): protocol: expected piece hash; context canceled; EOF Aug 10 23:05:07.699070750 (53h39m2.130150881s) [noise=true , address=104.36.82.61:4383 , piece_id=6ZNA5JKIQCDP4RH42V7AAADEYYDJKUARDHJJWBLHEHBK3POXRWQQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N24q6Li2xavkE61KntNxDPwQzTVHaG4LxyEqFUT1hhGYNQ9mp): protocol: rpc: dial tcp 104.36.82.61:4383: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:05:07.699179167 (53h39m2.130259288s) [noise=true , address=104.36.82.14:4247 , piece_id=QEHL3O75QFJ23FX3AZJSO2CBPFFYUELN3LLPZHFFQMZH4IPP2C2A, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1262xxX6aUPTYgKdvRxAyc77PmPBWJtq5cXRJFoELxHbfn18DLE): protocol: read tcp 188.166.225.170:60256->104.36.82.14:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:05:07.699591296 (53h39m2.130671427s) [noise=true , address=104.36.82.35:4583 , piece_id=AOQELQSBY44FQTFEOSAE4ZFRF7P55OKOD7DHICQXW6L57RQRYYGQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19): protocol: context canceled; context canceled Aug 10 23:05:07.699607436 (53h39m2.130687567s) [seg_pos={0 0}, upload=55940]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:05:07.699665718 (53h39m2.130745849s) [upload=55940]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:05:07.699669686 (53h39m2.130749807s) [upload=55940]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:05:08.009009212 (53h39m2.440089343s) [upload=55940]: Done uploading segments... Aug 10 23:05:08.009045151 (53h39m2.440125283s) [upload=55940]: Upload finished. err: <nil> Duration: 10.437942974s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X...........................................X....] duration: 1.845260927s FAIL (2025-08-10 23:06:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.845282448s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.437452979s FAIL (2025-08-10 23:06:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.437476635s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 7.382226881s FAIL (2025-08-10 23:06:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.382255627s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [...........X.......X.X.....X.X...........XX...............X.] duration: 10.254951835s FAIL (2025-08-10 23:05:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 10 23:05:03.554619361 (53h38m57.985699492s) [upload=55940]: Starting upload Aug 10 23:05:03.554665109 (53h38m57.98574524s) [upload=55940]: Done starting upload Aug 10 23:05:03.554811649 (53h38m57.98589179s) [upload=55940]: Uploading segments... Aug 10 23:05:03.554899368 (53h38m57.985979499s) [upload=55940]: Got next segment. Inline: false Aug 10 23:05:03.554964262 (53h38m57.986044393s) [upload=55940]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:05:03.554971365 (53h38m57.986051496s) [upload=55940]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:05:04.007203761 (53h38m58.438283892s) [seg_pos={0 0}, upload=55940]: Begin upload segment... Aug 10 23:05:04.007274326 (53h38m58.438354457s) [seg_pos={0 0}, upload=55940]: Done begin upload segment. Aug 10 23:05:04.007477475 (53h38m58.438557606s) [noise=true , address=104.36.82.12:4182 , piece_id=KXVG5327X5P3GMHMJW5UGL7PWMI2ZRQF6BXDWZXAUJ2AP7DKKK3A, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.007625788 (53h38m58.438705929s) [noise=true , address=104.36.82.4:4836 , piece_id=PAEI5LLGPCMAYKBKRTD7QXQGC6KDP4E6QCXLJYKBTMWZEM5IKXIA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.007960198 (53h38m58.439040329s) [noise=true , address=104.36.82.61:4383 , piece_id=6ZNA5JKIQCDP4RH42V7AAADEYYDJKUARDHJJWBLHEHBK3POXRWQQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.008859278 (53h38m58.43993941s) [noise=true , address=104.36.82.37:4663 , piece_id=RMC6ORT4SG3A7B3QB5L72ACQ3V5YY734BKIGRIXCWC5ADO6LBH3A, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.009284422 (53h38m58.440364584s) [noise=true , address=104.36.82.23:4457 , piece_id=OTRABU246BYQQMZKGWVNL7LFI2KLHQ3IGY6RR6CXIZL4M6TQDCVA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.009684147 (53h38m58.440764278s) [noise=true , address=45.91.116.9:4287 , piece_id=ZJGRIG6XWTNVG3GYBP5N4WJQLXYCPJW6JVLRMYA4CYX5SO7XKWYA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.010316126 (53h38m58.441396257s) [noise=true , address=104.36.82.58:4274 , piece_id=A7QKWC3YIMDBTIMOW5AKLE72I533AGAAOV66XEQEBANUTL72QBHQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.010614587 (53h38m58.441694718s) [noise=true , address=104.36.82.34:4570 , piece_id=J5CPBQHYKFZHZFZKPDLYG4G2P64P5MFTYYJVEK4NRN4MYBV2DEVA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.011014984 (53h38m58.442095115s) [noise=true , address=104.36.82.19:4407 , piece_id=M7VNWXU3KAKR6YV66VYBNRYICGT7PAIVMGINKIJJ6PNV2FB3RXSA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.011474642 (53h38m58.442554773s) [noise=true , address=104.36.82.35:4583 , piece_id=AOQELQSBY44FQTFEOSAE4ZFRF7P55OKOD7DHICQXW6L57RQRYYGQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.011921518 (53h38m58.443001649s) [noise=true , address=104.36.82.14:4247 , piece_id=QEHL3O75QFJ23FX3AZJSO2CBPFFYUELN3LLPZHFFQMZH4IPP2C2A, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.012318427 (53h38m58.443398558s) [noise=true , address=104.36.82.56:4231 , piece_id=RD4JOJGFE2CQZKDWIBBT56RRV2VDFDTQU4DLNZRMQTL6VOI422JA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.012784880 (53h38m58.443865011s) [noise=true , address=64.49.13.26:4947 , piece_id=CYBTXVHRWDPO54OFXDGQQAZ26YQS4DR7VT34POHNBRI3YFBYXJ7Q, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.014247087 (53h38m58.445327218s) [noise=true , address=104.36.82.38:4697 , piece_id=WX3FOUYDSWKPZW6D6WZFW4RXU2YIESB5OJTBPTH55EM5QB2JIMBQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.015466491 (53h38m58.446546622s) [noise=true , address=104.36.82.46:4499 , piece_id=2HPG6CKLRYXLYI6RQQWU2PHTESFLTH6ZVSKSILTX3OJLYVNTWSMQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.016638493 (53h38m58.447718624s) [noise=true , address=104.36.82.41:4777 , piece_id=UF5K4GTU2SW5LCHZ5APEDHJMSDMP64FA73LANW2VUE2GTCIOZMBA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.017984358 (53h38m58.449064489s) [noise=true , address=104.36.82.15:4269 , piece_id=JHKDOTBRLLH6YPR6VKOFVLPA3Q6YSMGUN4IF66ORDHS2W66FFYRQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.019254679 (53h38m58.45033482s) [noise=true , address=104.36.82.7:4936 , piece_id=2GTIP5HA7ZN4X3ACCD43TSVJT74NNIMWLPBVDYI7UIXQTBDKLD6Q, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:05.149856335 (53h38m59.580936466s) [upload=55940]: Next returned nil segment Aug 10 23:05:05.149871423 (53h38m59.580951554s) [upload=55940]: Waiting for error group managing segments... Aug 10 23:05:05.439410926 (53h38m59.870491057s) [noise=true , address=64.49.13.26:4947 , piece_id=CYBTXVHRWDPO54OFXDGQQAZ26YQS4DR7VT34POHNBRI3YFBYXJ7Q, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.051323200 (53h39m0.482403331s) [noise=true , address=104.36.82.56:4231 , piece_id=RD4JOJGFE2CQZKDWIBBT56RRV2VDFDTQU4DLNZRMQTL6VOI422JA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.075755667 (53h39m0.506835788s) [noise=true , address=104.36.82.7:4936 , piece_id=2GTIP5HA7ZN4X3ACCD43TSVJT74NNIMWLPBVDYI7UIXQTBDKLD6Q, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.097055631 (53h39m0.528135762s) [noise=true , address=104.36.82.46:4499 , piece_id=2HPG6CKLRYXLYI6RQQWU2PHTESFLTH6ZVSKSILTX3OJLYVNTWSMQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.098401586 (53h39m0.529481707s) [noise=true , address=104.36.82.34:4570 , piece_id=J5CPBQHYKFZHZFZKPDLYG4G2P64P5MFTYYJVEK4NRN4MYBV2DEVA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.125020757 (53h39m0.556100888s) [noise=true , address=104.36.82.58:4274 , piece_id=A7QKWC3YIMDBTIMOW5AKLE72I533AGAAOV66XEQEBANUTL72QBHQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.147088209 (53h39m0.57816838s) [noise=true , address=104.36.82.19:4407 , piece_id=M7VNWXU3KAKR6YV66VYBNRYICGT7PAIVMGINKIJJ6PNV2FB3RXSA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.161291916 (53h39m0.592372047s) [noise=true , address=104.36.82.23:4457 , piece_id=OTRABU246BYQQMZKGWVNL7LFI2KLHQ3IGY6RR6CXIZL4M6TQDCVA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.186986768 (53h39m0.618066889s) [noise=true , address=104.36.82.12:4182 , piece_id=KXVG5327X5P3GMHMJW5UGL7PWMI2ZRQF6BXDWZXAUJ2AP7DKKK3A, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.324393872 (53h39m0.755474003s) [noise=true , address=45.91.116.9:4287 , piece_id=ZJGRIG6XWTNVG3GYBP5N4WJQLXYCPJW6JVLRMYA4CYX5SO7XKWYA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.502418269 (53h39m0.9334984s) [noise=true , address=104.36.82.15:4269 , piece_id=JHKDOTBRLLH6YPR6VKOFVLPA3Q6YSMGUN4IF66ORDHS2W66FFYRQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.546654377 (53h39m0.977734509s) [noise=true , address=104.36.82.37:4663 , piece_id=RMC6ORT4SG3A7B3QB5L72ACQ3V5YY734BKIGRIXCWC5ADO6LBH3A, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.827166137 (53h39m1.258246268s) [noise=true , address=104.36.82.4:4836 , piece_id=PAEI5LLGPCMAYKBKRTD7QXQGC6KDP4E6QCXLJYKBTMWZEM5IKXIA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:07.698669291 (53h39m2.129749422s) [noise=true , address=104.36.82.38:4697 , piece_id=WX3FOUYDSWKPZW6D6WZFW4RXU2YIESB5OJTBPTH55EM5QB2JIMBQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:07.698700771 (53h39m2.129780892s) [seg_pos={0 0}, upload=55940]: Segment reached optimal threshold of 14 pieces. Aug 10 23:05:07.698835169 (53h39m2.1299153s) [noise=true , address=104.36.82.41:4777 , piece_id=UF5K4GTU2SW5LCHZ5APEDHJMSDMP64FA73LANW2VUE2GTCIOZMBA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Pvm1SftEF3tnCEBqUn1ZmyVcLDBkR6pyegEb7mtzjdQiiqYqt): protocol: expected piece hash; context canceled; EOF Aug 10 23:05:07.699070750 (53h39m2.130150881s) [noise=true , address=104.36.82.61:4383 , piece_id=6ZNA5JKIQCDP4RH42V7AAADEYYDJKUARDHJJWBLHEHBK3POXRWQQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N24q6Li2xavkE61KntNxDPwQzTVHaG4LxyEqFUT1hhGYNQ9mp): protocol: rpc: dial tcp 104.36.82.61:4383: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:05:07.699179167 (53h39m2.130259288s) [noise=true , address=104.36.82.14:4247 , piece_id=QEHL3O75QFJ23FX3AZJSO2CBPFFYUELN3LLPZHFFQMZH4IPP2C2A, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1262xxX6aUPTYgKdvRxAyc77PmPBWJtq5cXRJFoELxHbfn18DLE): protocol: read tcp 188.166.225.170:60256->104.36.82.14:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:05:07.699591296 (53h39m2.130671427s) [noise=true , address=104.36.82.35:4583 , piece_id=AOQELQSBY44FQTFEOSAE4ZFRF7P55OKOD7DHICQXW6L57RQRYYGQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19): protocol: context canceled; context canceled Aug 10 23:05:07.699607436 (53h39m2.130687567s) [seg_pos={0 0}, upload=55940]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:05:07.699665718 (53h39m2.130745849s) [upload=55940]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:05:07.699669686 (53h39m2.130749807s) [upload=55940]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:05:08.009009212 (53h39m2.440089343s) [upload=55940]: Done uploading segments... Aug 10 23:05:08.009045151 (53h39m2.440125283s) [upload=55940]: Upload finished. err: <nil> Duration: 10.254987584s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 10.413812534s FAIL (2025-08-10 23:05:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 10.413829796s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X...........................................X...] duration: 1.938062446s FAIL (2025-08-10 23:05:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.938093455s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 7.952289292s FAIL (2025-08-10 23:05:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.952324079s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.964531121s FAIL (2025-08-10 23:05:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 5.964548355s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X...........................................X..] duration: 1.836486286s FAIL (2025-08-10 23:05:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.836520131s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [............X.......X.X.....X.X...........XX...............X] duration: 5.564930329s FAIL (2025-08-10 23:05:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 10 23:05:03.554619361 (53h38m57.985699492s) [upload=55940]: Starting upload Aug 10 23:05:03.554665109 (53h38m57.98574524s) [upload=55940]: Done starting upload Aug 10 23:05:03.554811649 (53h38m57.98589179s) [upload=55940]: Uploading segments... Aug 10 23:05:03.554899368 (53h38m57.985979499s) [upload=55940]: Got next segment. Inline: false Aug 10 23:05:03.554964262 (53h38m57.986044393s) [upload=55940]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:05:03.554971365 (53h38m57.986051496s) [upload=55940]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:05:04.007203761 (53h38m58.438283892s) [seg_pos={0 0}, upload=55940]: Begin upload segment... Aug 10 23:05:04.007274326 (53h38m58.438354457s) [seg_pos={0 0}, upload=55940]: Done begin upload segment. Aug 10 23:05:04.007477475 (53h38m58.438557606s) [noise=true , address=104.36.82.12:4182 , piece_id=KXVG5327X5P3GMHMJW5UGL7PWMI2ZRQF6BXDWZXAUJ2AP7DKKK3A, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.007625788 (53h38m58.438705929s) [noise=true , address=104.36.82.4:4836 , piece_id=PAEI5LLGPCMAYKBKRTD7QXQGC6KDP4E6QCXLJYKBTMWZEM5IKXIA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.007960198 (53h38m58.439040329s) [noise=true , address=104.36.82.61:4383 , piece_id=6ZNA5JKIQCDP4RH42V7AAADEYYDJKUARDHJJWBLHEHBK3POXRWQQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.008859278 (53h38m58.43993941s) [noise=true , address=104.36.82.37:4663 , piece_id=RMC6ORT4SG3A7B3QB5L72ACQ3V5YY734BKIGRIXCWC5ADO6LBH3A, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.009284422 (53h38m58.440364584s) [noise=true , address=104.36.82.23:4457 , piece_id=OTRABU246BYQQMZKGWVNL7LFI2KLHQ3IGY6RR6CXIZL4M6TQDCVA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.009684147 (53h38m58.440764278s) [noise=true , address=45.91.116.9:4287 , piece_id=ZJGRIG6XWTNVG3GYBP5N4WJQLXYCPJW6JVLRMYA4CYX5SO7XKWYA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.010316126 (53h38m58.441396257s) [noise=true , address=104.36.82.58:4274 , piece_id=A7QKWC3YIMDBTIMOW5AKLE72I533AGAAOV66XEQEBANUTL72QBHQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.010614587 (53h38m58.441694718s) [noise=true , address=104.36.82.34:4570 , piece_id=J5CPBQHYKFZHZFZKPDLYG4G2P64P5MFTYYJVEK4NRN4MYBV2DEVA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.011014984 (53h38m58.442095115s) [noise=true , address=104.36.82.19:4407 , piece_id=M7VNWXU3KAKR6YV66VYBNRYICGT7PAIVMGINKIJJ6PNV2FB3RXSA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.011474642 (53h38m58.442554773s) [noise=true , address=104.36.82.35:4583 , piece_id=AOQELQSBY44FQTFEOSAE4ZFRF7P55OKOD7DHICQXW6L57RQRYYGQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.011921518 (53h38m58.443001649s) [noise=true , address=104.36.82.14:4247 , piece_id=QEHL3O75QFJ23FX3AZJSO2CBPFFYUELN3LLPZHFFQMZH4IPP2C2A, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.012318427 (53h38m58.443398558s) [noise=true , address=104.36.82.56:4231 , piece_id=RD4JOJGFE2CQZKDWIBBT56RRV2VDFDTQU4DLNZRMQTL6VOI422JA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.012784880 (53h38m58.443865011s) [noise=true , address=64.49.13.26:4947 , piece_id=CYBTXVHRWDPO54OFXDGQQAZ26YQS4DR7VT34POHNBRI3YFBYXJ7Q, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.014247087 (53h38m58.445327218s) [noise=true , address=104.36.82.38:4697 , piece_id=WX3FOUYDSWKPZW6D6WZFW4RXU2YIESB5OJTBPTH55EM5QB2JIMBQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.015466491 (53h38m58.446546622s) [noise=true , address=104.36.82.46:4499 , piece_id=2HPG6CKLRYXLYI6RQQWU2PHTESFLTH6ZVSKSILTX3OJLYVNTWSMQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.016638493 (53h38m58.447718624s) [noise=true , address=104.36.82.41:4777 , piece_id=UF5K4GTU2SW5LCHZ5APEDHJMSDMP64FA73LANW2VUE2GTCIOZMBA, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.017984358 (53h38m58.449064489s) [noise=true , address=104.36.82.15:4269 , piece_id=JHKDOTBRLLH6YPR6VKOFVLPA3Q6YSMGUN4IF66ORDHS2W66FFYRQ, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:04.019254679 (53h38m58.45033482s) [noise=true , address=104.36.82.7:4936 , piece_id=2GTIP5HA7ZN4X3ACCD43TSVJT74NNIMWLPBVDYI7UIXQTBDKLD6Q, seg_pos={0 0}, upload=55940]: Uploading piece... Aug 10 23:05:05.149856335 (53h38m59.580936466s) [upload=55940]: Next returned nil segment Aug 10 23:05:05.149871423 (53h38m59.580951554s) [upload=55940]: Waiting for error group managing segments... Aug 10 23:05:05.439410926 (53h38m59.870491057s) [noise=true , address=64.49.13.26:4947 , piece_id=CYBTXVHRWDPO54OFXDGQQAZ26YQS4DR7VT34POHNBRI3YFBYXJ7Q, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.051323200 (53h39m0.482403331s) [noise=true , address=104.36.82.56:4231 , piece_id=RD4JOJGFE2CQZKDWIBBT56RRV2VDFDTQU4DLNZRMQTL6VOI422JA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.075755667 (53h39m0.506835788s) [noise=true , address=104.36.82.7:4936 , piece_id=2GTIP5HA7ZN4X3ACCD43TSVJT74NNIMWLPBVDYI7UIXQTBDKLD6Q, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.097055631 (53h39m0.528135762s) [noise=true , address=104.36.82.46:4499 , piece_id=2HPG6CKLRYXLYI6RQQWU2PHTESFLTH6ZVSKSILTX3OJLYVNTWSMQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.098401586 (53h39m0.529481707s) [noise=true , address=104.36.82.34:4570 , piece_id=J5CPBQHYKFZHZFZKPDLYG4G2P64P5MFTYYJVEK4NRN4MYBV2DEVA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.125020757 (53h39m0.556100888s) [noise=true , address=104.36.82.58:4274 , piece_id=A7QKWC3YIMDBTIMOW5AKLE72I533AGAAOV66XEQEBANUTL72QBHQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.147088209 (53h39m0.57816838s) [noise=true , address=104.36.82.19:4407 , piece_id=M7VNWXU3KAKR6YV66VYBNRYICGT7PAIVMGINKIJJ6PNV2FB3RXSA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.161291916 (53h39m0.592372047s) [noise=true , address=104.36.82.23:4457 , piece_id=OTRABU246BYQQMZKGWVNL7LFI2KLHQ3IGY6RR6CXIZL4M6TQDCVA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.186986768 (53h39m0.618066889s) [noise=true , address=104.36.82.12:4182 , piece_id=KXVG5327X5P3GMHMJW5UGL7PWMI2ZRQF6BXDWZXAUJ2AP7DKKK3A, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.324393872 (53h39m0.755474003s) [noise=true , address=45.91.116.9:4287 , piece_id=ZJGRIG6XWTNVG3GYBP5N4WJQLXYCPJW6JVLRMYA4CYX5SO7XKWYA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.502418269 (53h39m0.9334984s) [noise=true , address=104.36.82.15:4269 , piece_id=JHKDOTBRLLH6YPR6VKOFVLPA3Q6YSMGUN4IF66ORDHS2W66FFYRQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.546654377 (53h39m0.977734509s) [noise=true , address=104.36.82.37:4663 , piece_id=RMC6ORT4SG3A7B3QB5L72ACQ3V5YY734BKIGRIXCWC5ADO6LBH3A, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:06.827166137 (53h39m1.258246268s) [noise=true , address=104.36.82.4:4836 , piece_id=PAEI5LLGPCMAYKBKRTD7QXQGC6KDP4E6QCXLJYKBTMWZEM5IKXIA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:07.698669291 (53h39m2.129749422s) [noise=true , address=104.36.82.38:4697 , piece_id=WX3FOUYDSWKPZW6D6WZFW4RXU2YIESB5OJTBPTH55EM5QB2JIMBQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: <nil> Aug 10 23:05:07.698700771 (53h39m2.129780892s) [seg_pos={0 0}, upload=55940]: Segment reached optimal threshold of 14 pieces. Aug 10 23:05:07.698835169 (53h39m2.1299153s) [noise=true , address=104.36.82.41:4777 , piece_id=UF5K4GTU2SW5LCHZ5APEDHJMSDMP64FA73LANW2VUE2GTCIOZMBA, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Pvm1SftEF3tnCEBqUn1ZmyVcLDBkR6pyegEb7mtzjdQiiqYqt): protocol: expected piece hash; context canceled; EOF Aug 10 23:05:07.699070750 (53h39m2.130150881s) [noise=true , address=104.36.82.61:4383 , piece_id=6ZNA5JKIQCDP4RH42V7AAADEYYDJKUARDHJJWBLHEHBK3POXRWQQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N24q6Li2xavkE61KntNxDPwQzTVHaG4LxyEqFUT1hhGYNQ9mp): protocol: rpc: dial tcp 104.36.82.61:4383: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:05:07.699179167 (53h39m2.130259288s) [noise=true , address=104.36.82.14:4247 , piece_id=QEHL3O75QFJ23FX3AZJSO2CBPFFYUELN3LLPZHFFQMZH4IPP2C2A, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1262xxX6aUPTYgKdvRxAyc77PmPBWJtq5cXRJFoELxHbfn18DLE): protocol: read tcp 188.166.225.170:60256->104.36.82.14:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:05:07.699591296 (53h39m2.130671427s) [noise=true , address=104.36.82.35:4583 , piece_id=AOQELQSBY44FQTFEOSAE4ZFRF7P55OKOD7DHICQXW6L57RQRYYGQ, seg_pos={0 0}, upload=55940]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19): protocol: context canceled; context canceled Aug 10 23:05:07.699607436 (53h39m2.130687567s) [seg_pos={0 0}, upload=55940]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:05:07.699665718 (53h39m2.130745849s) [upload=55940]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:05:07.699669686 (53h39m2.130749807s) [upload=55940]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:05:08.009009212 (53h39m2.440089343s) [upload=55940]: Done uploading segments... Aug 10 23:05:08.009045151 (53h39m2.440125283s) [upload=55940]: Upload finished. err: <nil> Duration: 5.565028887s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 8.280066437s FAIL (2025-08-10 23:04:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 8.280113819s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 10.959083952s FAIL (2025-08-10 23:04:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 10.959112227s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 7.559038102s FAIL (2025-08-10 23:04:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.559065745s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X...........................................X.] duration: 1.829182955s FAIL (2025-08-10 23:04:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.8292117s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.............X.......X.X.....X.X...........XX...............] duration: 10.314889536s FAIL (2025-08-10 23:04:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 10.314916596s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 7.809565341s FAIL (2025-08-10 23:03:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.809594026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X...........................................X] duration: 1.667910718s FAIL (2025-08-10 23:03:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.667957507s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [..............X.......X.X.....X.X...........XX..............] duration: 7.288233353s FAIL (2025-08-10 23:03:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 7.288256837s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 7.511406945s FAIL (2025-08-10 23:03:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.511431803s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.467528582s FAIL (2025-08-10 23:03:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.46755861s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 1.828855228s FAIL (2025-08-10 23:03:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.828883902s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.274954574s FAIL (2025-08-10 23:02:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.274980964s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.906502604s FAIL (2025-08-10 23:02:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.906532992s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 7.318016508s FAIL (2025-08-10 23:02:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.318043931s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 7.339176181s FAIL (2025-08-10 23:02:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.339205718s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [...............X.......X.X.....X.X...........XX.............] duration: 8.309667015s FAIL (2025-08-10 23:02:35 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 8.309694468s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.933543649s FAIL (2025-08-10 23:02:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.933571062s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 6.991116909s FAIL (2025-08-10 23:01:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.991197814s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [X...............X.......X.X.....X.X...........XX............] duration: 6.921202721s FAIL (2025-08-10 23:01:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 6.921233541s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 7.200311923s FAIL (2025-08-10 23:01:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.200338205s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 7.493345937s FAIL (2025-08-10 23:01:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.493376866s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.080384604s FAIL (2025-08-10 23:01:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.080400253s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X...............X.......X.X.....X.X...........XX...........] duration: 7.957286808s FAIL (2025-08-10 23:01:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 7.957316756s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [..X...............X.......X.X.....X.X...........XX..........] duration: 9.276873662s FAIL (2025-08-10 23:00:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 9.276899912s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.808500561s FAIL (2025-08-10 23:00:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.808533666s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 7.362975417s FAIL (2025-08-10 23:00:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.363001737s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.601244339s FAIL (2025-08-10 23:00:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.601386863s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [X..X...............X.......X.X.....X.X...........XX.........] duration: 9.042717924s FAIL (2025-08-10 23:00:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 9.042749504s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.968653315s FAIL (2025-08-10 23:00:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.968690516s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 7.280944585s FAIL (2025-08-10 22:59:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.280970233s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 7.45128957s FAIL (2025-08-10 22:59:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.451316149s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.179145018s FAIL (2025-08-10 22:59:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.179170767s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [.X..X...............X.......X.X.....X.X...........XX........] duration: 9.171350152s FAIL (2025-08-10 22:59:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 9.171374088s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.87765597s FAIL (2025-08-10 22:59:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.877673423s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 6.654253671s FAIL (2025-08-10 22:59:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.654282326s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.594205376s FAIL (2025-08-10 22:58:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.59423317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 7.017231453s FAIL (2025-08-10 22:58:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.017265597s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)