Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 7.447175871s FAIL (2025-08-10 23:21:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.447204665s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [....X.X...........XX...............X....X...................] duration: 8.567342634s FAIL (2025-08-10 23:21:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 8.567364056s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 11.421057046s FAIL (2025-08-10 23:21:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 11.421114225s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 7.709497071s FAIL (2025-08-10 23:21:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.709523441s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X..................X...] duration: 1.859339856s FAIL (2025-08-10 23:21:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 10 23:19:18.643569665 (53h53m13.074649796s) [upload=56195]: Starting upload Aug 10 23:19:18.643628578 (53h53m13.074708709s) [upload=56195]: Done starting upload Aug 10 23:19:18.643705515 (53h53m13.074785646s) [upload=56195]: Uploading segments... Aug 10 23:19:18.643802831 (53h53m13.074882962s) [upload=56195]: Got next segment. Inline: true Aug 10 23:19:18.643808132 (53h53m13.074888263s) [upload=56195]: Waiting for error group managing segments... Aug 10 23:19:18.643856193 (53h53m13.074936324s) [upload=56195]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:19:18.643861253 (53h53m13.074941384s) [upload=56195]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:19:18.643864640 (53h53m13.074944771s) [upload=56195]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:19:19.170086940 (53h53m13.601167071s) [upload=56195]: Done uploading segments... Aug 10 23:19:19.170104474 (53h53m13.601184605s) [upload=56195]: Upload finished. err: <nil> Duration:* 1.85936862s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 10.481231227s FAIL (2025-08-10 23:21:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 10.481257226s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X.X...........XX...............X....X..................] duration: 10.766657483s FAIL (2025-08-10 23:21:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 10.766683863s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................................X..................X..] duration: 1.830426477s FAIL (2025-08-10 23:20:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 10 23:19:18.643569665 (53h53m13.074649796s) [upload=56195]: Starting upload Aug 10 23:19:18.643628578 (53h53m13.074708709s) [upload=56195]: Done starting upload Aug 10 23:19:18.643705515 (53h53m13.074785646s) [upload=56195]: Uploading segments... Aug 10 23:19:18.643802831 (53h53m13.074882962s) [upload=56195]: Got next segment. Inline: true Aug 10 23:19:18.643808132 (53h53m13.074888263s) [upload=56195]: Waiting for error group managing segments... Aug 10 23:19:18.643856193 (53h53m13.074936324s) [upload=56195]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:19:18.643861253 (53h53m13.074941384s) [upload=56195]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:19:18.643864640 (53h53m13.074944771s) [upload=56195]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:19:19.170086940 (53h53m13.601167071s) [upload=56195]: Done uploading segments... Aug 10 23:19:19.170104474 (53h53m13.601184605s) [upload=56195]: Upload finished. err: <nil> Duration:* 1.830456483s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [X.....X.X...........XX...............X....X.................] duration: 7.50700559s FAIL (2025-08-10 23:20:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 7.507040177s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X.....X.X...........XX...............X....X................] duration: 7.864208626s FAIL (2025-08-10 23:20:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 7.864235589s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.98302008s FAIL (2025-08-10 23:20:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.983045176s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 7.475781409s FAIL (2025-08-10 23:19:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.475806737s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.250962305s FAIL (2025-08-10 23:19:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.250988214s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X..................X.] duration: 1.894403202s FAIL (2025-08-10 23:19:44 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 10 23:19:18.643569665 (53h53m13.074649796s) [upload=56195]: Starting upload Aug 10 23:19:18.643628578 (53h53m13.074708709s) [upload=56195]: Done starting upload Aug 10 23:19:18.643705515 (53h53m13.074785646s) [upload=56195]: Uploading segments... Aug 10 23:19:18.643802831 (53h53m13.074882962s) [upload=56195]: Got next segment. Inline: true Aug 10 23:19:18.643808132 (53h53m13.074888263s) [upload=56195]: Waiting for error group managing segments... Aug 10 23:19:18.643856193 (53h53m13.074936324s) [upload=56195]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:19:18.643861253 (53h53m13.074941384s) [upload=56195]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:19:18.643864640 (53h53m13.074944771s) [upload=56195]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:19:19.170086940 (53h53m13.601167071s) [upload=56195]: Done uploading segments... Aug 10 23:19:19.170104474 (53h53m13.601184605s) [upload=56195]: Upload finished. err: <nil> Duration:* 1.894426786s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................X..................X] duration: 1.859449818s FAIL (2025-08-10 23:19:19 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 10 23:19:18.643569665 (53h53m13.074649796s) [upload=56195]: Starting upload Aug 10 23:19:18.643628578 (53h53m13.074708709s) [upload=56195]: Done starting upload Aug 10 23:19:18.643705515 (53h53m13.074785646s) [upload=56195]: Uploading segments... Aug 10 23:19:18.643802831 (53h53m13.074882962s) [upload=56195]: Got next segment. Inline: true Aug 10 23:19:18.643808132 (53h53m13.074888263s) [upload=56195]: Waiting for error group managing segments... Aug 10 23:19:18.643856193 (53h53m13.074936324s) [upload=56195]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:19:18.643861253 (53h53m13.074941384s) [upload=56195]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:19:18.643864640 (53h53m13.074944771s) [upload=56195]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:19:19.170086940 (53h53m13.601167071s) [upload=56195]: Done uploading segments... Aug 10 23:19:19.170104474 (53h53m13.601184605s) [upload=56195]: Upload finished. err: <nil> Duration:* 1.859476991s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.X.....X.X...........XX...............X....X...............] duration: 9.562695644s FAIL (2025-08-10 23:19:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 9.562723127s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.628108199s FAIL (2025-08-10 23:19:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.628141282s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 8.016245891s FAIL (2025-08-10 23:18:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 8.016271682s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X.X.....X.X...........XX...............X....X..............] duration: 6.794812124s FAIL (2025-08-10 23:18:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 6.794841782s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 8.274211524s FAIL (2025-08-10 23:18:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 8.274238547s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 7.787751752s FAIL (2025-08-10 23:18:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.787776039s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.88379344s FAIL (2025-08-10 23:18:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.883832315s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [..X.X.....X.X...........XX...............X....X.............] duration: 9.941998793s FAIL (2025-08-10 23:18:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 9.942029953s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 8.091699746s FAIL (2025-08-10 23:17:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 8.091725926s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.03046646s FAIL (2025-08-10 23:17:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 2.030497058s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [...X.X.....X.X...........XX...............X....X............] duration: 10.372369606s FAIL (2025-08-10 23:17:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 10.372395755s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 10.273185986s FAIL (2025-08-10 23:17:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 10.273206094s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.849495428s FAIL (2025-08-10 23:17:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.849519875s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.875107595s FAIL (2025-08-10 23:16:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.875135769s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 8.325485048s FAIL (2025-08-10 23:16:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 8.325512191s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 7.463421071s FAIL (2025-08-10 23:16:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.463445197s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [....X.X.....X.X...........XX...............X....X...........] duration: 8.451124594s FAIL (2025-08-10 23:16:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 8.451150003s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 6.380001613s FAIL (2025-08-10 23:16:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.380024737s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 7.462152336s FAIL (2025-08-10 23:15:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.462178305s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [.....X.X.....X.X...........XX...............X....X..........] duration: 7.896642916s FAIL (2025-08-10 23:15:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 7.896675077s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [......X.X.....X.X...........XX...............X....X.........] duration: 9.810963377s FAIL (2025-08-10 23:15:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 9.810995939s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X...........................................X...............] duration: 1.863904208s FAIL (2025-08-10 23:15:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.863941828s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [.......X.X.....X.X...........XX...............X....X........] duration: 10.497846672s FAIL (2025-08-10 23:15:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 10.49787691s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X...........................................X..............] duration: 1.892330095s FAIL (2025-08-10 23:14:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.892361325s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 7.234791614s FAIL (2025-08-10 23:14:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.23482027s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 8.47022041s FAIL (2025-08-10 23:14:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 8.470239727s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [X.......X.X.....X.X...........XX...............X....X.......] duration: 8.702697737s FAIL (2025-08-10 23:14:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 8.702724328s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.467879443s FAIL (2025-08-10 23:14:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.467897807s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X...........................................X.............] duration: 1.986989386s FAIL (2025-08-10 23:14:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.98701668s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 7.212399543s FAIL (2025-08-10 23:13:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.212427025s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [.X.......X.X.....X.X...........XX...............X....X......] duration: 9.224501897s FAIL (2025-08-10 23:13:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 23:09:14.954968192 (53h43m9.386048323s) [upload=56014]: Starting upload Aug 10 23:09:14.955017124 (53h43m9.386097245s) [upload=56014]: Done starting upload Aug 10 23:09:14.955137054 (53h43m9.386217186s) [upload=56014]: Uploading segments... Aug 10 23:09:14.955229551 (53h43m9.386309682s) [upload=56014]: Got next segment. Inline: false Aug 10 23:09:14.955262334 (53h43m9.386342465s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:09:14.955266872 (53h43m9.386347003s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 23:09:15.493877597 (53h43m9.924957729s) [seg_pos={0 0}, upload=56014]: Begin upload segment... Aug 10 23:09:15.493927032 (53h43m9.925007163s) [seg_pos={0 0}, upload=56014]: Done begin upload segment. Aug 10 23:09:15.493952179 (53h43m9.925032311s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494000743 (53h43m9.925080894s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494380640 (53h43m9.925460771s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494545806 (53h43m9.925625937s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.494666406 (53h43m9.925746537s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495019862 (53h43m9.926099983s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495300108 (53h43m9.926380239s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495441249 (53h43m9.92652138s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495616635 (53h43m9.926696766s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.495871422 (53h43m9.926951553s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496141439 (53h43m9.92722157s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496285414 (53h43m9.927365545s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496544250 (53h43m9.927624381s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.496686502 (53h43m9.927766633s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497106956 (53h43m9.928187097s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497123017 (53h43m9.928203148s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497519846 (53h43m9.928599977s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:15.497572647 (53h43m9.928652778s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Uploading piece... Aug 10 23:09:16.602750874 (53h43m11.033831006s) [upload=56014]: Next returned nil segment Aug 10 23:09:16.602767496 (53h43m11.033847627s) [upload=56014]: Waiting for error group managing segments... Aug 10 23:09:16.902595006 (53h43m11.333675137s) [noise=true , address=64.49.13.27:4964 , piece_id=MUP7K3BOR4SPFZDW2XKF57PH6NN7DMLB4BIBALZOSI7E5HS767YA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.924303100 (53h43m11.355383221s) [noise=true , address=64.49.13.26:4949 , piece_id=LXIKAVDY6Z7Y7MDKBPMPWDFHHPB4TOCBW7HK3HZBDM75JA672OCA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.925022306 (53h43m11.356102437s) [noise=true , address=79.127.205.234:4613 , piece_id=OUCEPNOI5URRBCR23CDS7DQMV5LWSZTFMRSQPKBMZ5LBNAOWW5CA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.949241304 (53h43m11.380321435s) [noise=true , address=64.49.13.26:4951 , piece_id=BMFGONGPPQ2ZFCLEHCLRBBPUUCMHRCKKUSVVVEDIW55IBEBCOV4Q, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:16.955589631 (53h43m11.386669762s) [noise=true , address=64.49.13.169:4706 , piece_id=5BV2JDXH6L5WYAPSWAXXQXPFY7JXAFTAMY5K4Y3WX7FIWSAPYOXA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.468604351 (53h43m11.899684482s) [noise=true , address=104.36.82.43:4862 , piece_id=RYNMBWPU75HXBDVCWMFTDICTD4ND7DZX6WNYIMV5RXY4PVLWORFQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.526815486 (53h43m11.957895617s) [noise=true , address=104.36.82.38:4719 , piece_id=HAETFK5DGZ7F5FIQYXS3XJP26JJCPKQSH26TPBPU3XI53UHI5GCQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.527881857 (53h43m11.958961988s) [noise=true , address=79.127.201.215:4355 , piece_id=D4GWM3QAR4QBS74BVPAMTTFTTVYN4UZOROKME6Z5VSBW2LAPMTIQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.544336099 (53h43m11.97541623s) [noise=true , address=104.36.82.12:4179 , piece_id=A6M5DYVJBBUJTOV6VAQL6SJPIF4VZPALZ6DSGGALTR2HCIARQBQA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.559416172 (53h43m11.990496303s) [noise=true , address=104.36.82.62:4416 , piece_id=VQ62GGQHHA7E3AEM7A2U5VOMEQ2RPU5VDAV7HCUI7ZBOZSZF565A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.603181127 (53h43m12.034261258s) [noise=true , address=104.36.82.39:4740 , piece_id=2YFWDAITCR6ISKUPCFGAPNG5G5MIPA6BEL6P3TA43MAFACMMKY7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.693220549 (53h43m12.12430067s) [noise=true , address=104.36.82.19:4415 , piece_id=EXJQFAYWAR2UWQ4G2PFEXDI3BFVVYRDTZWBOWTAHA5LGKKG6PQ4A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.710626261 (53h43m12.141706393s) [noise=true , address=104.36.82.61:4378 , piece_id=J3G24ZW4P2FHFYGWNQIGHB6W57I5EDRJHVZV4LHZHFURXAIF56NQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747857736 (53h43m12.178937867s) [noise=true , address=104.36.82.2:4756 , piece_id=GUCV6TKRHBYPPBZX2ETJBDHNGQJWLSACGRPMXC5YJ5ZGKN3ZUZEQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: <nil> Aug 10 23:09:17.747882173 (53h43m12.178962304s) [seg_pos={0 0}, upload=56014]: Segment reached optimal threshold of 14 pieces. Aug 10 23:09:17.748048792 (53h43m12.179128923s) [noise=true , address=104.36.82.42:4815 , piece_id=NMC47HT5W7LATH2SAVTVJ3Q2A7F2WY2KK7OMZEUS6CEYJDJM6ORQ, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX): protocol: expected piece hash; context canceled; EOF Aug 10 23:09:17.748312236 (53h43m12.179392368s) [noise=true , address=104.36.82.36:4643 , piece_id=YVONOPGP23IIYFSHX4QZ6LM5RVVXKLSQPX57F3GOUK7W4PZIKRYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:53966->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748336202 (53h43m12.179416334s) [noise=true , address=104.36.82.47:4521 , piece_id=RYT4TRMTHHQIICQ762GSBARUP6C733VE3OM3PZI3DLSIM4VBEG7A, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD): protocol: rpc: dial tcp 104.36.82.47:4521: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748369155 (53h43m12.179449286s) [noise=true , address=104.36.82.41:4792 , piece_id=3JNUAIMKQXW5NHZES644RAFP5M5LVA2MM2QF3OZV3LHMUQIFQXYA, seg_pos={0 0}, upload=56014]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k): protocol: write tcp 188.166.225.170:46266->104.36.82.41:4792: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 23:09:17.748388953 (53h43m12.179469085s) [seg_pos={0 0}, upload=56014]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 23:09:17.748445131 (53h43m12.179525262s) [upload=56014]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 23:09:17.748448998 (53h43m12.179529129s) [upload=56014]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:09:18.092446969 (53h43m12.52352709s) [upload=56014]: Done uploading segments... Aug 10 23:09:18.092466526 (53h43m12.523546647s) [upload=56014]: Upload finished. err: <nil> Duration: 9.224526063s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 8.122505629s FAIL (2025-08-10 23:13:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 8.122525286s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 7.504002662s FAIL (2025-08-10 23:13:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.504030646s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X...........................................X............] duration: 2.002490546s FAIL (2025-08-10 23:13:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 2.002516276s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 10.085438819s FAIL (2025-08-10 23:13:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 10.085463606s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)