Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.....X......................] duration: 2.119800882s FAIL (2025-08-16 16:54:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.119826672s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X..............X..............X.] duration: 6.185057224s FAIL (2025-08-16 16:53:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:53:00.920598464 (191h26m55.351678595s) [upload=199148]: Starting upload Aug 16 16:53:00.920641066 (191h26m55.351721187s) [upload=199148]: Done starting upload Aug 16 16:53:00.920734364 (191h26m55.351814495s) [upload=199148]: Uploading segments... Aug 16 16:53:00.920801493 (191h26m55.351881624s) [upload=199148]: Got next segment. Inline: false Aug 16 16:53:00.920829507 (191h26m55.351909638s) [upload=199148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:53:00.920833835 (191h26m55.351913966s) [upload=199148]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:53:01.444199383 (191h26m55.875279514s) [seg_pos={0 0}, upload=199148]: Begin upload segment... Aug 16 16:53:01.444302420 (191h26m55.875382611s) [seg_pos={0 0}, upload=199148]: Done begin upload segment. Aug 16 16:53:01.444345423 (191h26m55.875425604s) [noise=true , address=79.127.220.101:4072 , piece_id=YJFEGMKE6TY4ZAPZEZ5AJCYFODVE3ADW7CLOAA6DL4PI6ZBGJTWQ, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.444395498 (191h26m55.875475629s) [noise=true , address=104.36.82.9:4020 , piece_id=C56XVUQMZNZ3INLXYLYD2GM27OMJBOINKH627UCOKYD5GHIQOCKA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.445020153 (191h26m55.876100285s) [noise=true , address=77.90.31.10:4529 , piece_id=WLDMXHVYYTWGIVUZJDW3TW3ZMFKKLF5RI3P3TVXT5RVQ555BIX7Q, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.445056693 (191h26m55.876136824s) [noise=true , address=45.91.116.13:4186 , piece_id=A4VXENTNLARO3M7TT6SUJJN22Y63QOVSJ7AJHBTUDXBYRKWCRJDA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.445521633 (191h26m55.876601764s) [noise=true , address=77.90.31.19:4848 , piece_id=RMBPNGLAGQFJLC6I5JOCZCBYB5JZHYEVTISHT7ITGXX47CQ32ZLA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.445599792 (191h26m55.876679933s) [noise=true , address=77.90.31.18:4817 , piece_id=FKAMFAFRGA6LZ7IJIFU5GY3JT5UJEW7V2RPQUE2JVKHY4MBYSDFQ, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.446003736 (191h26m55.877083867s) [noise=true , address=77.90.31.13:4658 , piece_id=GENI2QIK3CASIKYRWWHINDPZHYHMAKHUZNT55MWCWLTT24IG6SYA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.446248103 (191h26m55.877328234s) [noise=true , address=77.90.31.21:4906 , piece_id=CVDG7KBZDQUSOSY4UV3WBAU6ALFA74S2TZLUANHGDPN4FK24PZKA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.446518280 (191h26m55.877598421s) [noise=true , address=77.90.31.11:4566 , piece_id=CRFH3OLMSZSWP6G775ZQKSEPXZVXFHF7CZRRMWJUUI7GLV2ZNGYQ, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.446734594 (191h26m55.877814725s) [noise=true , address=77.90.31.15:4729 , piece_id=WAW2RH6L2RV53C42BLGBMIUZ3TMK7FJX43M2QSJ43KXUEHDUYS7Q, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.446953383 (191h26m55.878033524s) [noise=true , address=45.91.116.6:4005 , piece_id=NTMQECGZV72N52XLIXQIBDKZWZWWABBDBWT2GGYRW3WGQPBEHRNA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.447266331 (191h26m55.878346462s) [noise=true , address=104.36.82.41:4795 , piece_id=MDV4SLN7OVDOWFIWRFMB4BNQX6HVXTM2IBX256SB27E27C4TFOEA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.447565865 (191h26m55.878645996s) [noise=true , address=45.91.116.14:4201 , piece_id=D25W5E63RXQFBUQTU3S4UIVDHSXEH7HKRX7BK6J6GKXSEGW7K5PA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.447643894 (191h26m55.878724015s) [noise=true , address=77.90.31.16:4750 , piece_id=YIOVGWVQMPMUR76ORZOYHZRCAMGM5AD7GQGYHF7CJY7BOLAFF37A, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.447868433 (191h26m55.878948565s) [noise=true , address=104.36.82.58:4291 , piece_id=Z2IVVDGHBAKPL7V7QM35HYMDQCBMXVNKUKXQRQ3MFGR237UCRQTQ, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.448101258 (191h26m55.87918139s) [noise=true , address=104.36.82.23:4458 , piece_id=NATJSURGW3OG4ARRG3OCT32XFXSG6GT27RM2QMPRIGDGZ62XHUJA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.448358401 (191h26m55.879438542s) [noise=true , address=104.36.82.36:4626 , piece_id=57HZF6U63DKKFPFKL6RQUBZJKMWKLZUUS27AVK6Q4X6NR6LQ5J5A, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.448471267 (191h26m55.879551408s) [noise=true , address=104.36.82.8:4002 , piece_id=XS566VVKIJSJ5RSEEQE5OHV5TI6B7DG3UJRVUVOXSJR4E36MXZ4A, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:02.916082029 (191h26m57.34716216s) [upload=199148]: Next returned nil segment Aug 16 16:53:02.916098019 (191h26m57.34717815s) [upload=199148]: Waiting for error group managing segments... Aug 16 16:53:03.318560545 (191h26m57.749640676s) [noise=true , address=79.127.220.101:4072 , piece_id=YJFEGMKE6TY4ZAPZEZ5AJCYFODVE3ADW7CLOAA6DL4PI6ZBGJTWQ, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.353522491 (191h26m57.784602612s) [noise=true , address=77.90.31.11:4566 , piece_id=CRFH3OLMSZSWP6G775ZQKSEPXZVXFHF7CZRRMWJUUI7GLV2ZNGYQ, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.394436284 (191h26m57.825516415s) [noise=true , address=104.36.82.8:4002 , piece_id=XS566VVKIJSJ5RSEEQE5OHV5TI6B7DG3UJRVUVOXSJR4E36MXZ4A, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.431244373 (191h26m57.862324494s) [noise=true , address=77.90.31.16:4750 , piece_id=YIOVGWVQMPMUR76ORZOYHZRCAMGM5AD7GQGYHF7CJY7BOLAFF37A, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.475163496 (191h26m57.906243627s) [noise=true , address=77.90.31.21:4906 , piece_id=CVDG7KBZDQUSOSY4UV3WBAU6ALFA74S2TZLUANHGDPN4FK24PZKA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.500379024 (191h26m57.931459155s) [noise=true , address=77.90.31.15:4729 , piece_id=WAW2RH6L2RV53C42BLGBMIUZ3TMK7FJX43M2QSJ43KXUEHDUYS7Q, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.534194725 (191h26m57.965274856s) [noise=true , address=104.36.82.23:4458 , piece_id=NATJSURGW3OG4ARRG3OCT32XFXSG6GT27RM2QMPRIGDGZ62XHUJA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.541829174 (191h26m57.972909305s) [noise=true , address=104.36.82.41:4795 , piece_id=MDV4SLN7OVDOWFIWRFMB4BNQX6HVXTM2IBX256SB27E27C4TFOEA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.561350675 (191h26m57.992430806s) [noise=true , address=45.91.116.6:4005 , piece_id=NTMQECGZV72N52XLIXQIBDKZWZWWABBDBWT2GGYRW3WGQPBEHRNA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.567285682 (191h26m57.998365803s) [noise=true , address=104.36.82.36:4626 , piece_id=57HZF6U63DKKFPFKL6RQUBZJKMWKLZUUS27AVK6Q4X6NR6LQ5J5A, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.569142654 (191h26m58.000222785s) [noise=true , address=77.90.31.13:4658 , piece_id=GENI2QIK3CASIKYRWWHINDPZHYHMAKHUZNT55MWCWLTT24IG6SYA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.591525201 (191h26m58.022605322s) [noise=true , address=77.90.31.10:4529 , piece_id=WLDMXHVYYTWGIVUZJDW3TW3ZMFKKLF5RI3P3TVXT5RVQ555BIX7Q, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.592193188 (191h26m58.023273319s) [noise=true , address=77.90.31.19:4848 , piece_id=RMBPNGLAGQFJLC6I5JOCZCBYB5JZHYEVTISHT7ITGXX47CQ32ZLA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.600141397 (191h26m58.031221528s) [noise=true , address=104.36.82.9:4020 , piece_id=C56XVUQMZNZ3INLXYLYD2GM27OMJBOINKH627UCOKYD5GHIQOCKA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.600168959 (191h26m58.03124909s) [seg_pos={0 0}, upload=199148]: Segment reached optimal threshold of 14 pieces. Aug 16 16:53:03.600513379 (191h26m58.03159351s) [noise=true , address=45.91.116.13:4186 , piece_id=A4VXENTNLARO3M7TT6SUJJN22Y63QOVSJ7AJHBTUDXBYRKWCRJDA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U2z7FPoTM2387UEPKbaP1NFas5APdPkUzuA2D6BwjKxC57hht): protocol: expected piece hash; context canceled; EOF Aug 16 16:53:03.600592901 (191h26m58.031673032s) [noise=true , address=77.90.31.18:4817 , piece_id=FKAMFAFRGA6LZ7IJIFU5GY3JT5UJEW7V2RPQUE2JVKHY4MBYSDFQ, seg_pos={0 0}, upload=199148]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qv5LBuANP9vS2vbsVNrJ7Mkc3SqXwTNCBzLvN955RHFKQVv5x): protocol: write tcp 188.166.225.170:51644->77.90.31.18:4817: use of closed network connection; write tcp 188.166.225.170:51644->77.90.31.18:4817: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51644->77.90.31.18:4817: use of closed network connection Aug 16 16:53:03.600772594 (191h26m58.031852725s) [noise=true , address=45.91.116.14:4201 , piece_id=D25W5E63RXQFBUQTU3S4UIVDHSXEH7HKRX7BK6J6GKXSEGW7K5PA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RUqtJWMppPz7u7bTYm5chvDf53HVDPCwC7bnmTs1R5hJD8Chp): protocol: write tcp 188.166.225.170:56346->45.91.116.14:4201: use of closed network connection; write tcp 188.166.225.170:56346->45.91.116.14:4201: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56346->45.91.116.14:4201: use of closed network connection Aug 16 16:53:03.600911821 (191h26m58.031991952s) [noise=true , address=104.36.82.58:4291 , piece_id=Z2IVVDGHBAKPL7V7QM35HYMDQCBMXVNKUKXQRQ3MFGR237UCRQTQ, seg_pos={0 0}, upload=199148]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: context canceled; context canceled Aug 16 16:53:03.600923524 (191h26m58.032003655s) [seg_pos={0 0}, upload=199148]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:53:03.601111794 (191h26m58.032191945s) [upload=199148]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:53:03.601118386 (191h26m58.032198517s) [upload=199148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:53:04.867116055 (191h26m59.298196186s) [upload=199148]: Done uploading segments... Aug 16 16:53:04.867133088 (191h26m59.298213209s) [upload=199148]: Upload finished. err: <nil> Duration: 6.185087071s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X............................X..............X..............X] duration: 5.319182325s FAIL (2025-08-16 16:53:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:53:00.920598464 (191h26m55.351678595s) [upload=199148]: Starting upload Aug 16 16:53:00.920641066 (191h26m55.351721187s) [upload=199148]: Done starting upload Aug 16 16:53:00.920734364 (191h26m55.351814495s) [upload=199148]: Uploading segments... Aug 16 16:53:00.920801493 (191h26m55.351881624s) [upload=199148]: Got next segment. Inline: false Aug 16 16:53:00.920829507 (191h26m55.351909638s) [upload=199148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:53:00.920833835 (191h26m55.351913966s) [upload=199148]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:53:01.444199383 (191h26m55.875279514s) [seg_pos={0 0}, upload=199148]: Begin upload segment... Aug 16 16:53:01.444302420 (191h26m55.875382611s) [seg_pos={0 0}, upload=199148]: Done begin upload segment. Aug 16 16:53:01.444345423 (191h26m55.875425604s) [noise=true , address=79.127.220.101:4072 , piece_id=YJFEGMKE6TY4ZAPZEZ5AJCYFODVE3ADW7CLOAA6DL4PI6ZBGJTWQ, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.444395498 (191h26m55.875475629s) [noise=true , address=104.36.82.9:4020 , piece_id=C56XVUQMZNZ3INLXYLYD2GM27OMJBOINKH627UCOKYD5GHIQOCKA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.445020153 (191h26m55.876100285s) [noise=true , address=77.90.31.10:4529 , piece_id=WLDMXHVYYTWGIVUZJDW3TW3ZMFKKLF5RI3P3TVXT5RVQ555BIX7Q, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.445056693 (191h26m55.876136824s) [noise=true , address=45.91.116.13:4186 , piece_id=A4VXENTNLARO3M7TT6SUJJN22Y63QOVSJ7AJHBTUDXBYRKWCRJDA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.445521633 (191h26m55.876601764s) [noise=true , address=77.90.31.19:4848 , piece_id=RMBPNGLAGQFJLC6I5JOCZCBYB5JZHYEVTISHT7ITGXX47CQ32ZLA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.445599792 (191h26m55.876679933s) [noise=true , address=77.90.31.18:4817 , piece_id=FKAMFAFRGA6LZ7IJIFU5GY3JT5UJEW7V2RPQUE2JVKHY4MBYSDFQ, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.446003736 (191h26m55.877083867s) [noise=true , address=77.90.31.13:4658 , piece_id=GENI2QIK3CASIKYRWWHINDPZHYHMAKHUZNT55MWCWLTT24IG6SYA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.446248103 (191h26m55.877328234s) [noise=true , address=77.90.31.21:4906 , piece_id=CVDG7KBZDQUSOSY4UV3WBAU6ALFA74S2TZLUANHGDPN4FK24PZKA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.446518280 (191h26m55.877598421s) [noise=true , address=77.90.31.11:4566 , piece_id=CRFH3OLMSZSWP6G775ZQKSEPXZVXFHF7CZRRMWJUUI7GLV2ZNGYQ, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.446734594 (191h26m55.877814725s) [noise=true , address=77.90.31.15:4729 , piece_id=WAW2RH6L2RV53C42BLGBMIUZ3TMK7FJX43M2QSJ43KXUEHDUYS7Q, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.446953383 (191h26m55.878033524s) [noise=true , address=45.91.116.6:4005 , piece_id=NTMQECGZV72N52XLIXQIBDKZWZWWABBDBWT2GGYRW3WGQPBEHRNA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.447266331 (191h26m55.878346462s) [noise=true , address=104.36.82.41:4795 , piece_id=MDV4SLN7OVDOWFIWRFMB4BNQX6HVXTM2IBX256SB27E27C4TFOEA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.447565865 (191h26m55.878645996s) [noise=true , address=45.91.116.14:4201 , piece_id=D25W5E63RXQFBUQTU3S4UIVDHSXEH7HKRX7BK6J6GKXSEGW7K5PA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.447643894 (191h26m55.878724015s) [noise=true , address=77.90.31.16:4750 , piece_id=YIOVGWVQMPMUR76ORZOYHZRCAMGM5AD7GQGYHF7CJY7BOLAFF37A, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.447868433 (191h26m55.878948565s) [noise=true , address=104.36.82.58:4291 , piece_id=Z2IVVDGHBAKPL7V7QM35HYMDQCBMXVNKUKXQRQ3MFGR237UCRQTQ, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.448101258 (191h26m55.87918139s) [noise=true , address=104.36.82.23:4458 , piece_id=NATJSURGW3OG4ARRG3OCT32XFXSG6GT27RM2QMPRIGDGZ62XHUJA, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.448358401 (191h26m55.879438542s) [noise=true , address=104.36.82.36:4626 , piece_id=57HZF6U63DKKFPFKL6RQUBZJKMWKLZUUS27AVK6Q4X6NR6LQ5J5A, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:01.448471267 (191h26m55.879551408s) [noise=true , address=104.36.82.8:4002 , piece_id=XS566VVKIJSJ5RSEEQE5OHV5TI6B7DG3UJRVUVOXSJR4E36MXZ4A, seg_pos={0 0}, upload=199148]: Uploading piece... Aug 16 16:53:02.916082029 (191h26m57.34716216s) [upload=199148]: Next returned nil segment Aug 16 16:53:02.916098019 (191h26m57.34717815s) [upload=199148]: Waiting for error group managing segments... Aug 16 16:53:03.318560545 (191h26m57.749640676s) [noise=true , address=79.127.220.101:4072 , piece_id=YJFEGMKE6TY4ZAPZEZ5AJCYFODVE3ADW7CLOAA6DL4PI6ZBGJTWQ, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.353522491 (191h26m57.784602612s) [noise=true , address=77.90.31.11:4566 , piece_id=CRFH3OLMSZSWP6G775ZQKSEPXZVXFHF7CZRRMWJUUI7GLV2ZNGYQ, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.394436284 (191h26m57.825516415s) [noise=true , address=104.36.82.8:4002 , piece_id=XS566VVKIJSJ5RSEEQE5OHV5TI6B7DG3UJRVUVOXSJR4E36MXZ4A, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.431244373 (191h26m57.862324494s) [noise=true , address=77.90.31.16:4750 , piece_id=YIOVGWVQMPMUR76ORZOYHZRCAMGM5AD7GQGYHF7CJY7BOLAFF37A, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.475163496 (191h26m57.906243627s) [noise=true , address=77.90.31.21:4906 , piece_id=CVDG7KBZDQUSOSY4UV3WBAU6ALFA74S2TZLUANHGDPN4FK24PZKA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.500379024 (191h26m57.931459155s) [noise=true , address=77.90.31.15:4729 , piece_id=WAW2RH6L2RV53C42BLGBMIUZ3TMK7FJX43M2QSJ43KXUEHDUYS7Q, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.534194725 (191h26m57.965274856s) [noise=true , address=104.36.82.23:4458 , piece_id=NATJSURGW3OG4ARRG3OCT32XFXSG6GT27RM2QMPRIGDGZ62XHUJA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.541829174 (191h26m57.972909305s) [noise=true , address=104.36.82.41:4795 , piece_id=MDV4SLN7OVDOWFIWRFMB4BNQX6HVXTM2IBX256SB27E27C4TFOEA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.561350675 (191h26m57.992430806s) [noise=true , address=45.91.116.6:4005 , piece_id=NTMQECGZV72N52XLIXQIBDKZWZWWABBDBWT2GGYRW3WGQPBEHRNA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.567285682 (191h26m57.998365803s) [noise=true , address=104.36.82.36:4626 , piece_id=57HZF6U63DKKFPFKL6RQUBZJKMWKLZUUS27AVK6Q4X6NR6LQ5J5A, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.569142654 (191h26m58.000222785s) [noise=true , address=77.90.31.13:4658 , piece_id=GENI2QIK3CASIKYRWWHINDPZHYHMAKHUZNT55MWCWLTT24IG6SYA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.591525201 (191h26m58.022605322s) [noise=true , address=77.90.31.10:4529 , piece_id=WLDMXHVYYTWGIVUZJDW3TW3ZMFKKLF5RI3P3TVXT5RVQ555BIX7Q, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.592193188 (191h26m58.023273319s) [noise=true , address=77.90.31.19:4848 , piece_id=RMBPNGLAGQFJLC6I5JOCZCBYB5JZHYEVTISHT7ITGXX47CQ32ZLA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.600141397 (191h26m58.031221528s) [noise=true , address=104.36.82.9:4020 , piece_id=C56XVUQMZNZ3INLXYLYD2GM27OMJBOINKH627UCOKYD5GHIQOCKA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: <nil> Aug 16 16:53:03.600168959 (191h26m58.03124909s) [seg_pos={0 0}, upload=199148]: Segment reached optimal threshold of 14 pieces. Aug 16 16:53:03.600513379 (191h26m58.03159351s) [noise=true , address=45.91.116.13:4186 , piece_id=A4VXENTNLARO3M7TT6SUJJN22Y63QOVSJ7AJHBTUDXBYRKWCRJDA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U2z7FPoTM2387UEPKbaP1NFas5APdPkUzuA2D6BwjKxC57hht): protocol: expected piece hash; context canceled; EOF Aug 16 16:53:03.600592901 (191h26m58.031673032s) [noise=true , address=77.90.31.18:4817 , piece_id=FKAMFAFRGA6LZ7IJIFU5GY3JT5UJEW7V2RPQUE2JVKHY4MBYSDFQ, seg_pos={0 0}, upload=199148]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qv5LBuANP9vS2vbsVNrJ7Mkc3SqXwTNCBzLvN955RHFKQVv5x): protocol: write tcp 188.166.225.170:51644->77.90.31.18:4817: use of closed network connection; write tcp 188.166.225.170:51644->77.90.31.18:4817: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51644->77.90.31.18:4817: use of closed network connection Aug 16 16:53:03.600772594 (191h26m58.031852725s) [noise=true , address=45.91.116.14:4201 , piece_id=D25W5E63RXQFBUQTU3S4UIVDHSXEH7HKRX7BK6J6GKXSEGW7K5PA, seg_pos={0 0}, upload=199148]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RUqtJWMppPz7u7bTYm5chvDf53HVDPCwC7bnmTs1R5hJD8Chp): protocol: write tcp 188.166.225.170:56346->45.91.116.14:4201: use of closed network connection; write tcp 188.166.225.170:56346->45.91.116.14:4201: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56346->45.91.116.14:4201: use of closed network connection Aug 16 16:53:03.600911821 (191h26m58.031991952s) [noise=true , address=104.36.82.58:4291 , piece_id=Z2IVVDGHBAKPL7V7QM35HYMDQCBMXVNKUKXQRQ3MFGR237UCRQTQ, seg_pos={0 0}, upload=199148]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gn31FU1BnbHPLvB7k1RZhtJbDdJCLyBLa5A6VhDNMFN7erSG1): protocol: context canceled; context canceled Aug 16 16:53:03.600923524 (191h26m58.032003655s) [seg_pos={0 0}, upload=199148]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:53:03.601111794 (191h26m58.032191945s) [upload=199148]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:53:03.601118386 (191h26m58.032198517s) [upload=199148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:53:04.867116055 (191h26m59.298196186s) [upload=199148]: Done uploading segments... Aug 16 16:53:04.867133088 (191h26m59.298213209s) [upload=199148]: Upload finished. err: <nil> Duration: 5.319208896s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X.....X.....................] duration: 2.039830346s FAIL (2025-08-16 16:53:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.039856005s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X............................X..............X..............] duration: 6.405736777s FAIL (2025-08-16 16:52:51 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 6.405754009s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.....X....................] duration: 2.439230386s FAIL (2025-08-16 16:52:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.439256085s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X............................X..............X.............] duration: 6.701546538s FAIL (2025-08-16 16:52:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 6.701574002s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.....X...................] duration: 1.976816869s FAIL (2025-08-16 16:51:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 1.976836686s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X............................X..............X............] duration: 6.083526819s FAIL (2025-08-16 16:51:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 6.083598927s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.....X..................] duration: 2.232838454s FAIL (2025-08-16 16:51:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.232867852s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.....X.................] duration: 1.92335787s FAIL (2025-08-16 16:50:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 1.923376406s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X............................X..............X...........] duration: 6.662380067s FAIL (2025-08-16 16:50:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 6.662470218s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.....X................] duration: 1.945246954s FAIL (2025-08-16 16:49:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 1.945268896s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X............................X..............X..........] duration: 6.171495857s FAIL (2025-08-16 16:49:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 6.171515624s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X.....................................X.....X...............] duration: 2.136177429s FAIL (2025-08-16 16:48:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.136206824s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X............................X..............X.........] duration: 6.112728365s FAIL (2025-08-16 16:48:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 6.112751159s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X.....................................X.....X..............] duration: 2.063760468s FAIL (2025-08-16 16:48:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.06379287s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X............................X..............X........] duration: 7.449662869s FAIL (2025-08-16 16:48:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 7.44968926s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X.....................................X.....X.............] duration: 2.029657253s FAIL (2025-08-16 16:47:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.029685617s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.......X............................X..............X.......] duration: 6.517677149s FAIL (2025-08-16 16:47:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 6.517703551s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [XX.......X............................X..............X......] duration: 7.221736587s FAIL (2025-08-16 16:47:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 7.221757958s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X.....................................X.....X............] duration: 2.483814878s FAIL (2025-08-16 16:47:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.483848141s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....X.....................................X.....X...........] duration: 2.057500169s FAIL (2025-08-16 16:46:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.057520557s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [XXX.......X............................X..............X.....] duration: 6.22322792s FAIL (2025-08-16 16:46:01 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 6.223252989s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....X.....................................X.....X..........] duration: 3.032614702s FAIL (2025-08-16 16:45:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 3.032637236s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [XXXX.......X............................X..............X....] duration: 8.373484648s FAIL (2025-08-16 16:44:51 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 8.373511749s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......X.....................................X.....X.........] duration: 2.991984014s FAIL (2025-08-16 16:44:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.992015664s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.990439036s FAIL (2025-08-16 16:43:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== . Aug 16 15:59:57.963918156 (190h33m52.394998287s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964029729 (190h33m52.39510986s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964451335 (190h33m52.395531467s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964727965 (190h33m52.395808086s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964797899 (190h33m52.39587802s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965042367 (190h33m52.396122498s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965090028 (190h33m52.396170159s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965305831 (190h33m52.396385962s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965343212 (190h33m52.396423344s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965548745 (190h33m52.396628876s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965569796 (190h33m52.396649927s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965765831 (190h33m52.396845962s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966003606 (190h33m52.397083737s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966271599 (190h33m52.39735173s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:59.736319114 (190h33m54.167399245s) [upload=198240]: Next returned nil segment Aug 16 15:59:59.736337630 (190h33m54.167417761s) [upload=198240]: Waiting for error group managing segments... Aug 16 16:00:00.028564434 (190h33m54.459644565s) [noise=true , address=104.36.82.6:4874 , piece_id=FI3EZZH5G2O42GFDQGX6ZRLX2KFM55UTNMCJY7AGYS2LPH4WCO3Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.092359868 (190h33m54.523439999s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.158899656 (190h33m54.589979787s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.274239525 (190h33m54.705319646s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:08.612621871 (190h34m3.043702012s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:25.194912489 (190h34m19.62599263s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:27.035229138 (190h34m21.466309269s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035398122 (190h34m21.466478263s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035640335 (190h34m21.466720496s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection Aug 16 16:00:27.035680382 (190h34m21.466760513s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection Aug 16 16:00:27.035716621 (190h34m21.466796752s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection Aug 16 16:00:27.035748352 (190h34m21.466828483s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: rpc: dial tcp 104.36.82.16:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035901174 (190h34m21.466981305s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 188.166.225.170:33266->104.36.82.11:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035939287 (190h34m21.467019418s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection Aug 16 16:00:27.035966249 (190h34m21.46704638s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection Aug 16 16:00:27.036250663 (190h34m21.467330794s) [noise=true , address=104.36.82.8:4970 , piece_id=XIPQ7G4S7ETV46ME3HDJTUUXDTNGWS2ZX75VO57NV4R35OGLFZOA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: context deadline exceeded; context deadline exceeded Aug 16 16:00:27.036322741 (190h34m21.467402872s) [seg_pos={0 0}, upload=198240]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 16 16:00:27.438563272 (190h34m21.869643404s) [upload=198240]: Done uploading segments... Aug 16 16:00:27.438589974 (190h34m21.869670105s) [upload=198240]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.990467531s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [XXXXX.......X............................X..............X...] duration: 8.255220122s FAIL (2025-08-16 16:43:52 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 8.255249038s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......X.....................................X.....X........] duration: 3.519277958s FAIL (2025-08-16 16:43:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 3.519322863s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.166015579s FAIL (2025-08-16 16:43:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== upload=198245]: Uploading piece... Aug 16 16:00:10.770806034 (190h34m5.201886165s) [noise=true , address=104.36.82.58:4271 , piece_id=M5B7V2YGDPVNZL73XKJH2YKBAGZR7LCKRLCKQGIBMABSRLAHMSWA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.770975228 (190h34m5.202055359s) [noise=true , address=104.36.82.60:4353 , piece_id=R6OTTIHWGZE2KAS25Z6YCTCWL2RAQKZGTDFI66Y4JYPSK72I7TTA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771077693 (190h34m5.202157825s) [noise=true , address=77.90.31.14:4695 , piece_id=ZE67SDK22KFKKAONRYDWRRMG266Q24DHKNVLFNE3NTZDI6XILQSQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771274441 (190h34m5.202354572s) [noise=true , address=104.36.82.45:4934 , piece_id=6AMSA3INP3L3ADURWGJDGXTFOFMQ5VSORF3KNGO35ZSPQNYMSP2Q, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771309197 (190h34m5.202389328s) [noise=true , address=104.36.82.44:4888 , piece_id=TPTSTGF3IAALD6BRRPENCRWLDCWNBEUFB5T6P64Q733SPGL6YOKA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771570006 (190h34m5.202650137s) [noise=true , address=104.36.82.5:4842 , piece_id=DXZQKVGVMH7TGKVZKRFLI7LTEJY7OBQX6GKRAM4T33CZVRJNGX3A, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771594673 (190h34m5.202674804s) [noise=true , address=79.127.220.101:4076 , piece_id=LJU4XJG4OFORQVRTD7BOOI47GREELV5F5BGHNZ3VUBBP3HS73JYQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771829703 (190h34m5.202909834s) [noise=true , address=104.36.82.36:4629 , piece_id=K3L4U3YD5N6WDP35N2QKPZ3KLNSCR5R4QWEZS4DMQO4YXOBYXCYQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771875119 (190h34m5.20295527s) [noise=true , address=45.91.116.7:4043 , piece_id=2V7JUZMBVKV6IFF6TSHFUTJLVYC7ZWC3AG2F3RXRYZUGWCVVNO6A, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772062949 (190h34m5.20314308s) [noise=true , address=77.90.31.11:4559 , piece_id=IXHIHQXDLTSVFOPDCAHGYDEI22RBH77SU2DJFZ52MLWZMTANVADQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772183270 (190h34m5.203263401s) [noise=true , address=104.36.82.22:4543 , piece_id=XTN6FE7SKOCWPE6VQTEIFZFZ6OLPFSJSMJRUNYQUU4DJ7JCZR5UQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772563297 (190h34m5.203643428s) [noise=true , address=104.36.82.16:4306 , piece_id=2547BUZR4R5PGGF7MHB2HIWIX42JLKHM4AMNFTSUGYIM55YYYBHQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:11.953464880 (190h34m6.384545011s) [upload=198245]: Next returned nil segment Aug 16 16:00:11.953482854 (190h34m6.384562985s) [upload=198245]: Waiting for error group managing segments... Aug 16 16:00:12.941409550 (190h34m7.372489691s) [noise=true , address=79.127.220.101:4076 , piece_id=LJU4XJG4OFORQVRTD7BOOI47GREELV5F5BGHNZ3VUBBP3HS73JYQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:13.659968773 (190h34m8.091048904s) [noise=true , address=45.91.116.7:4043 , piece_id=2V7JUZMBVKV6IFF6TSHFUTJLVYC7ZWC3AG2F3RXRYZUGWCVVNO6A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1TkHPinJgbiX4PmRfNH7wRxtDZ9rASSvbZtLxZuxbF3UaoSqFD, address:45.91.116.7:4043): protocol: storage node overloaded, request limit: 2000 Aug 16 16:00:14.611558252 (190h34m9.042638383s) [noise=true , address=104.36.82.45:4934 , piece_id=6AMSA3INP3L3ADURWGJDGXTFOFMQ5VSORF3KNGO35ZSPQNYMSP2Q, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:20.670138101 (190h34m15.101218242s) [noise=true , address=66.23.193.175:4557 , piece_id=RDX5DR364JZMUSTWW4HA4P55RNDONJXR676CFUMOG5TO5SJ7JQQA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:22.111492140 (190h34m16.542572311s) [noise=true , address=45.91.116.5:4315 , piece_id=DYNZCXOWKSP6KPULJLUTQHS6JPGPX7WVCESHEH6TTQQIXZM3QZZA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:24.242804738 (190h34m18.673884869s) [noise=true , address=77.90.31.18:4837 , piece_id=5TSI4MM4SHIYCL3PCD3RHYCJR4FXPM7VVVXQNU7ROAZNWHEON4DQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:43.969823882 (190h34m38.400904103s) [noise=true , address=77.90.31.14:4695 , piece_id=ZE67SDK22KFKKAONRYDWRRMG266Q24DHKNVLFNE3NTZDI6XILQSQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:01:09.606812307 (190h35m4.037892438s) [noise=true , address=104.36.82.5:4842 , piece_id=DXZQKVGVMH7TGKVZKRFLI7LTEJY7OBQX6GKRAM4T33CZVRJNGX3A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn, address:104.36.82.5:4842): protocol: rpc: dial tcp 104.36.82.5:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606854288 (190h35m4.037934419s) [noise=true , address=104.36.82.58:4271 , piece_id=M5B7V2YGDPVNZL73XKJH2YKBAGZR7LCKRLCKQGIBMABSRLAHMSWA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1, address:104.36.82.58:4271): protocol: rpc: dial tcp 104.36.82.58:4271: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606884385 (190h35m4.037964516s) [noise=true , address=104.36.82.60:4353 , piece_id=R6OTTIHWGZE2KAS25Z6YCTCWL2RAQKZGTDFI66Y4JYPSK72I7TTA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12BmzVLKbkqPMB7bJHGzFbgRRLu3TLg6TdTBCvYybsvz2yLpBbQ, address:104.36.82.60:4353): protocol: rpc: dial tcp 104.36.82.60:4353: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606917428 (190h35m4.037997559s) [noise=true , address=104.36.82.23:4457 , piece_id=DCL3MYX7AVNV2V3KD5DHFDYYLY3AT6OXE3FEHOGIJB4WLCHQ5KOQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12mpuVB9gpUQ6dNLaMAVaDkgctVSetSbPRxhxU49WhAzycwwNhs, address:104.36.82.23:4457): protocol: write tcp 188.166.225.170:39772->104.36.82.23:4457: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606949459 (190h35m4.03802959s) [noise=true , address=104.36.82.36:4629 , piece_id=K3L4U3YD5N6WDP35N2QKPZ3KLNSCR5R4QWEZS4DMQO4YXOBYXCYQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1uxMhTiH446Ku7h1PutqVh7fSCzwKHR7oypYBzGWELLNP7ojf6, address:104.36.82.36:4629): protocol: write tcp 188.166.225.170:59572->104.36.82.36:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607050022 (190h35m4.038130153s) [noise=true , address=104.36.82.3:4799 , piece_id=LIEZASMYJIYFGFSUGAYBQNS5OY6UPGJ7KWCTBWVMLZ5YMRPGEMTQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:104.36.82.3:4799): protocol: write tcp 188.166.225.170:37514->104.36.82.3:4799: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607085500 (190h35m4.038165631s) [noise=true , address=77.90.31.19:4867 , piece_id=WFPEU7FO65IMB3JAFZ6P3IPIEEOXAW6QDPGK7VRXFLWTSEQMC52A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1gczGyJMCtqUfaac5oy7o4CTwypVF8AZjrAAjoJweU3kBFbuXw, address:77.90.31.19:4867): protocol: write tcp 188.166.225.170:59224->77.90.31.19:4867: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607125927 (190h35m4.038206059s) [noise=true , address=104.36.82.44:4888 , piece_id=TPTSTGF3IAALD6BRRPENCRWLDCWNBEUFB5T6P64Q733SPGL6YOKA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:129FkuTqtaF3DmzHFvcCuJRvaubQV6HbaV1CnFtoY95QHH6NcME, address:104.36.82.44:4888): protocol: rpc: dial tcp 104.36.82.44:4888: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607138701 (190h35m4.038218832s) [noise=true , address=77.90.31.11:4559 , piece_id=IXHIHQXDLTSVFOPDCAHGYDEI22RBH77SU2DJFZ52MLWZMTANVADQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection; write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection Aug 16 16:01:09.607166785 (190h35m4.038246916s) [noise=true , address=104.36.82.22:4543 , piece_id=XTN6FE7SKOCWPE6VQTEIFZFZ6OLPFSJSMJRUNYQUU4DJ7JCZR5UQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:19iUYyo8cGDLGDnvdNo2Qfj1K6tp6EJWF5U6rXD4gJ7Hv7Gsch, address:104.36.82.22:4543): protocol: rpc: dial tcp 104.36.82.22:4543: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607185561 (190h35m4.038265692s) [noise=true , address=104.36.82.16:4306 , piece_id=2547BUZR4R5PGGF7MHB2HIWIX42JLKHM4AMNFTSUGYIM55YYYBHQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1mcaP7H2gd9gWpbcPK79gvemz5L6k5UgbwQ5t6aaTU7nXut1p7, address:104.36.82.16:4306): protocol: rpc: dial tcp 104.36.82.16:4306: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607211200 (190h35m4.038291332s) [seg_pos={0 0}, upload=198245]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 16 16:01:09.843745933 (190h35m4.274826064s) [upload=198245]: Done uploading segments... Aug 16 16:01:09.843772213 (190h35m4.274852344s) [upload=198245]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 6.166043472s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 6.831721527s FAIL (2025-08-16 16:43:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== . Aug 16 15:59:57.963918156 (190h33m52.394998287s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964029729 (190h33m52.39510986s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964451335 (190h33m52.395531467s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964727965 (190h33m52.395808086s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964797899 (190h33m52.39587802s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965042367 (190h33m52.396122498s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965090028 (190h33m52.396170159s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965305831 (190h33m52.396385962s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965343212 (190h33m52.396423344s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965548745 (190h33m52.396628876s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965569796 (190h33m52.396649927s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965765831 (190h33m52.396845962s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966003606 (190h33m52.397083737s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966271599 (190h33m52.39735173s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:59.736319114 (190h33m54.167399245s) [upload=198240]: Next returned nil segment Aug 16 15:59:59.736337630 (190h33m54.167417761s) [upload=198240]: Waiting for error group managing segments... Aug 16 16:00:00.028564434 (190h33m54.459644565s) [noise=true , address=104.36.82.6:4874 , piece_id=FI3EZZH5G2O42GFDQGX6ZRLX2KFM55UTNMCJY7AGYS2LPH4WCO3Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.092359868 (190h33m54.523439999s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.158899656 (190h33m54.589979787s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.274239525 (190h33m54.705319646s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:08.612621871 (190h34m3.043702012s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:25.194912489 (190h34m19.62599263s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:27.035229138 (190h34m21.466309269s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035398122 (190h34m21.466478263s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035640335 (190h34m21.466720496s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection Aug 16 16:00:27.035680382 (190h34m21.466760513s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection Aug 16 16:00:27.035716621 (190h34m21.466796752s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection Aug 16 16:00:27.035748352 (190h34m21.466828483s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: rpc: dial tcp 104.36.82.16:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035901174 (190h34m21.466981305s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 188.166.225.170:33266->104.36.82.11:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035939287 (190h34m21.467019418s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection Aug 16 16:00:27.035966249 (190h34m21.46704638s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection Aug 16 16:00:27.036250663 (190h34m21.467330794s) [noise=true , address=104.36.82.8:4970 , piece_id=XIPQ7G4S7ETV46ME3HDJTUUXDTNGWS2ZX75VO57NV4R35OGLFZOA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: context deadline exceeded; context deadline exceeded Aug 16 16:00:27.036322741 (190h34m21.467402872s) [seg_pos={0 0}, upload=198240]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 16 16:00:27.438563272 (190h34m21.869643404s) [upload=198240]: Done uploading segments... Aug 16 16:00:27.438589974 (190h34m21.869670105s) [upload=198240]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.831751153s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [XXXXXX.......X............................X..............X..] duration: 7.766559471s FAIL (2025-08-16 16:43:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 7.766612191s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........X.....................................X.....X.......] duration: 3.218453334s FAIL (2025-08-16 16:43:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 3.218480126s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 5.750809634s FAIL (2025-08-16 16:43:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== . Aug 16 15:59:57.963918156 (190h33m52.394998287s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964029729 (190h33m52.39510986s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964451335 (190h33m52.395531467s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964727965 (190h33m52.395808086s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964797899 (190h33m52.39587802s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965042367 (190h33m52.396122498s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965090028 (190h33m52.396170159s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965305831 (190h33m52.396385962s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965343212 (190h33m52.396423344s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965548745 (190h33m52.396628876s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965569796 (190h33m52.396649927s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965765831 (190h33m52.396845962s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966003606 (190h33m52.397083737s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966271599 (190h33m52.39735173s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:59.736319114 (190h33m54.167399245s) [upload=198240]: Next returned nil segment Aug 16 15:59:59.736337630 (190h33m54.167417761s) [upload=198240]: Waiting for error group managing segments... Aug 16 16:00:00.028564434 (190h33m54.459644565s) [noise=true , address=104.36.82.6:4874 , piece_id=FI3EZZH5G2O42GFDQGX6ZRLX2KFM55UTNMCJY7AGYS2LPH4WCO3Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.092359868 (190h33m54.523439999s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.158899656 (190h33m54.589979787s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.274239525 (190h33m54.705319646s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:08.612621871 (190h34m3.043702012s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:25.194912489 (190h34m19.62599263s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:27.035229138 (190h34m21.466309269s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035398122 (190h34m21.466478263s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035640335 (190h34m21.466720496s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection Aug 16 16:00:27.035680382 (190h34m21.466760513s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection Aug 16 16:00:27.035716621 (190h34m21.466796752s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection Aug 16 16:00:27.035748352 (190h34m21.466828483s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: rpc: dial tcp 104.36.82.16:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035901174 (190h34m21.466981305s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 188.166.225.170:33266->104.36.82.11:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035939287 (190h34m21.467019418s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection Aug 16 16:00:27.035966249 (190h34m21.46704638s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection Aug 16 16:00:27.036250663 (190h34m21.467330794s) [noise=true , address=104.36.82.8:4970 , piece_id=XIPQ7G4S7ETV46ME3HDJTUUXDTNGWS2ZX75VO57NV4R35OGLFZOA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: context deadline exceeded; context deadline exceeded Aug 16 16:00:27.036322741 (190h34m21.467402872s) [seg_pos={0 0}, upload=198240]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 16 16:00:27.438563272 (190h34m21.869643404s) [upload=198240]: Done uploading segments... Aug 16 16:00:27.438589974 (190h34m21.869670105s) [upload=198240]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.750842045s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 6.221782878s FAIL (2025-08-16 16:43:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== upload=198245]: Uploading piece... Aug 16 16:00:10.770806034 (190h34m5.201886165s) [noise=true , address=104.36.82.58:4271 , piece_id=M5B7V2YGDPVNZL73XKJH2YKBAGZR7LCKRLCKQGIBMABSRLAHMSWA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.770975228 (190h34m5.202055359s) [noise=true , address=104.36.82.60:4353 , piece_id=R6OTTIHWGZE2KAS25Z6YCTCWL2RAQKZGTDFI66Y4JYPSK72I7TTA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771077693 (190h34m5.202157825s) [noise=true , address=77.90.31.14:4695 , piece_id=ZE67SDK22KFKKAONRYDWRRMG266Q24DHKNVLFNE3NTZDI6XILQSQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771274441 (190h34m5.202354572s) [noise=true , address=104.36.82.45:4934 , piece_id=6AMSA3INP3L3ADURWGJDGXTFOFMQ5VSORF3KNGO35ZSPQNYMSP2Q, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771309197 (190h34m5.202389328s) [noise=true , address=104.36.82.44:4888 , piece_id=TPTSTGF3IAALD6BRRPENCRWLDCWNBEUFB5T6P64Q733SPGL6YOKA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771570006 (190h34m5.202650137s) [noise=true , address=104.36.82.5:4842 , piece_id=DXZQKVGVMH7TGKVZKRFLI7LTEJY7OBQX6GKRAM4T33CZVRJNGX3A, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771594673 (190h34m5.202674804s) [noise=true , address=79.127.220.101:4076 , piece_id=LJU4XJG4OFORQVRTD7BOOI47GREELV5F5BGHNZ3VUBBP3HS73JYQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771829703 (190h34m5.202909834s) [noise=true , address=104.36.82.36:4629 , piece_id=K3L4U3YD5N6WDP35N2QKPZ3KLNSCR5R4QWEZS4DMQO4YXOBYXCYQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771875119 (190h34m5.20295527s) [noise=true , address=45.91.116.7:4043 , piece_id=2V7JUZMBVKV6IFF6TSHFUTJLVYC7ZWC3AG2F3RXRYZUGWCVVNO6A, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772062949 (190h34m5.20314308s) [noise=true , address=77.90.31.11:4559 , piece_id=IXHIHQXDLTSVFOPDCAHGYDEI22RBH77SU2DJFZ52MLWZMTANVADQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772183270 (190h34m5.203263401s) [noise=true , address=104.36.82.22:4543 , piece_id=XTN6FE7SKOCWPE6VQTEIFZFZ6OLPFSJSMJRUNYQUU4DJ7JCZR5UQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772563297 (190h34m5.203643428s) [noise=true , address=104.36.82.16:4306 , piece_id=2547BUZR4R5PGGF7MHB2HIWIX42JLKHM4AMNFTSUGYIM55YYYBHQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:11.953464880 (190h34m6.384545011s) [upload=198245]: Next returned nil segment Aug 16 16:00:11.953482854 (190h34m6.384562985s) [upload=198245]: Waiting for error group managing segments... Aug 16 16:00:12.941409550 (190h34m7.372489691s) [noise=true , address=79.127.220.101:4076 , piece_id=LJU4XJG4OFORQVRTD7BOOI47GREELV5F5BGHNZ3VUBBP3HS73JYQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:13.659968773 (190h34m8.091048904s) [noise=true , address=45.91.116.7:4043 , piece_id=2V7JUZMBVKV6IFF6TSHFUTJLVYC7ZWC3AG2F3RXRYZUGWCVVNO6A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1TkHPinJgbiX4PmRfNH7wRxtDZ9rASSvbZtLxZuxbF3UaoSqFD, address:45.91.116.7:4043): protocol: storage node overloaded, request limit: 2000 Aug 16 16:00:14.611558252 (190h34m9.042638383s) [noise=true , address=104.36.82.45:4934 , piece_id=6AMSA3INP3L3ADURWGJDGXTFOFMQ5VSORF3KNGO35ZSPQNYMSP2Q, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:20.670138101 (190h34m15.101218242s) [noise=true , address=66.23.193.175:4557 , piece_id=RDX5DR364JZMUSTWW4HA4P55RNDONJXR676CFUMOG5TO5SJ7JQQA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:22.111492140 (190h34m16.542572311s) [noise=true , address=45.91.116.5:4315 , piece_id=DYNZCXOWKSP6KPULJLUTQHS6JPGPX7WVCESHEH6TTQQIXZM3QZZA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:24.242804738 (190h34m18.673884869s) [noise=true , address=77.90.31.18:4837 , piece_id=5TSI4MM4SHIYCL3PCD3RHYCJR4FXPM7VVVXQNU7ROAZNWHEON4DQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:43.969823882 (190h34m38.400904103s) [noise=true , address=77.90.31.14:4695 , piece_id=ZE67SDK22KFKKAONRYDWRRMG266Q24DHKNVLFNE3NTZDI6XILQSQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:01:09.606812307 (190h35m4.037892438s) [noise=true , address=104.36.82.5:4842 , piece_id=DXZQKVGVMH7TGKVZKRFLI7LTEJY7OBQX6GKRAM4T33CZVRJNGX3A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn, address:104.36.82.5:4842): protocol: rpc: dial tcp 104.36.82.5:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606854288 (190h35m4.037934419s) [noise=true , address=104.36.82.58:4271 , piece_id=M5B7V2YGDPVNZL73XKJH2YKBAGZR7LCKRLCKQGIBMABSRLAHMSWA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1, address:104.36.82.58:4271): protocol: rpc: dial tcp 104.36.82.58:4271: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606884385 (190h35m4.037964516s) [noise=true , address=104.36.82.60:4353 , piece_id=R6OTTIHWGZE2KAS25Z6YCTCWL2RAQKZGTDFI66Y4JYPSK72I7TTA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12BmzVLKbkqPMB7bJHGzFbgRRLu3TLg6TdTBCvYybsvz2yLpBbQ, address:104.36.82.60:4353): protocol: rpc: dial tcp 104.36.82.60:4353: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606917428 (190h35m4.037997559s) [noise=true , address=104.36.82.23:4457 , piece_id=DCL3MYX7AVNV2V3KD5DHFDYYLY3AT6OXE3FEHOGIJB4WLCHQ5KOQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12mpuVB9gpUQ6dNLaMAVaDkgctVSetSbPRxhxU49WhAzycwwNhs, address:104.36.82.23:4457): protocol: write tcp 188.166.225.170:39772->104.36.82.23:4457: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606949459 (190h35m4.03802959s) [noise=true , address=104.36.82.36:4629 , piece_id=K3L4U3YD5N6WDP35N2QKPZ3KLNSCR5R4QWEZS4DMQO4YXOBYXCYQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1uxMhTiH446Ku7h1PutqVh7fSCzwKHR7oypYBzGWELLNP7ojf6, address:104.36.82.36:4629): protocol: write tcp 188.166.225.170:59572->104.36.82.36:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607050022 (190h35m4.038130153s) [noise=true , address=104.36.82.3:4799 , piece_id=LIEZASMYJIYFGFSUGAYBQNS5OY6UPGJ7KWCTBWVMLZ5YMRPGEMTQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:104.36.82.3:4799): protocol: write tcp 188.166.225.170:37514->104.36.82.3:4799: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607085500 (190h35m4.038165631s) [noise=true , address=77.90.31.19:4867 , piece_id=WFPEU7FO65IMB3JAFZ6P3IPIEEOXAW6QDPGK7VRXFLWTSEQMC52A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1gczGyJMCtqUfaac5oy7o4CTwypVF8AZjrAAjoJweU3kBFbuXw, address:77.90.31.19:4867): protocol: write tcp 188.166.225.170:59224->77.90.31.19:4867: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607125927 (190h35m4.038206059s) [noise=true , address=104.36.82.44:4888 , piece_id=TPTSTGF3IAALD6BRRPENCRWLDCWNBEUFB5T6P64Q733SPGL6YOKA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:129FkuTqtaF3DmzHFvcCuJRvaubQV6HbaV1CnFtoY95QHH6NcME, address:104.36.82.44:4888): protocol: rpc: dial tcp 104.36.82.44:4888: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607138701 (190h35m4.038218832s) [noise=true , address=77.90.31.11:4559 , piece_id=IXHIHQXDLTSVFOPDCAHGYDEI22RBH77SU2DJFZ52MLWZMTANVADQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection; write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection Aug 16 16:01:09.607166785 (190h35m4.038246916s) [noise=true , address=104.36.82.22:4543 , piece_id=XTN6FE7SKOCWPE6VQTEIFZFZ6OLPFSJSMJRUNYQUU4DJ7JCZR5UQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:19iUYyo8cGDLGDnvdNo2Qfj1K6tp6EJWF5U6rXD4gJ7Hv7Gsch, address:104.36.82.22:4543): protocol: rpc: dial tcp 104.36.82.22:4543: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607185561 (190h35m4.038265692s) [noise=true , address=104.36.82.16:4306 , piece_id=2547BUZR4R5PGGF7MHB2HIWIX42JLKHM4AMNFTSUGYIM55YYYBHQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1mcaP7H2gd9gWpbcPK79gvemz5L6k5UgbwQ5t6aaTU7nXut1p7, address:104.36.82.16:4306): protocol: rpc: dial tcp 104.36.82.16:4306: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607211200 (190h35m4.038291332s) [seg_pos={0 0}, upload=198245]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 16 16:01:09.843745933 (190h35m4.274826064s) [upload=198245]: Done uploading segments... Aug 16 16:01:09.843772213 (190h35m4.274852344s) [upload=198245]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 6.221813526s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXXXXXX.......X............................X..............X.] duration: 6.791066816s FAIL (2025-08-16 16:42:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 6.791090882s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........X.....................................X.....X......] duration: 3.269697474s FAIL (2025-08-16 16:42:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 3.269728374s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 6.167076332s FAIL (2025-08-16 16:42:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== upload=198245]: Uploading piece... Aug 16 16:00:10.770806034 (190h34m5.201886165s) [noise=true , address=104.36.82.58:4271 , piece_id=M5B7V2YGDPVNZL73XKJH2YKBAGZR7LCKRLCKQGIBMABSRLAHMSWA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.770975228 (190h34m5.202055359s) [noise=true , address=104.36.82.60:4353 , piece_id=R6OTTIHWGZE2KAS25Z6YCTCWL2RAQKZGTDFI66Y4JYPSK72I7TTA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771077693 (190h34m5.202157825s) [noise=true , address=77.90.31.14:4695 , piece_id=ZE67SDK22KFKKAONRYDWRRMG266Q24DHKNVLFNE3NTZDI6XILQSQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771274441 (190h34m5.202354572s) [noise=true , address=104.36.82.45:4934 , piece_id=6AMSA3INP3L3ADURWGJDGXTFOFMQ5VSORF3KNGO35ZSPQNYMSP2Q, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771309197 (190h34m5.202389328s) [noise=true , address=104.36.82.44:4888 , piece_id=TPTSTGF3IAALD6BRRPENCRWLDCWNBEUFB5T6P64Q733SPGL6YOKA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771570006 (190h34m5.202650137s) [noise=true , address=104.36.82.5:4842 , piece_id=DXZQKVGVMH7TGKVZKRFLI7LTEJY7OBQX6GKRAM4T33CZVRJNGX3A, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771594673 (190h34m5.202674804s) [noise=true , address=79.127.220.101:4076 , piece_id=LJU4XJG4OFORQVRTD7BOOI47GREELV5F5BGHNZ3VUBBP3HS73JYQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771829703 (190h34m5.202909834s) [noise=true , address=104.36.82.36:4629 , piece_id=K3L4U3YD5N6WDP35N2QKPZ3KLNSCR5R4QWEZS4DMQO4YXOBYXCYQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771875119 (190h34m5.20295527s) [noise=true , address=45.91.116.7:4043 , piece_id=2V7JUZMBVKV6IFF6TSHFUTJLVYC7ZWC3AG2F3RXRYZUGWCVVNO6A, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772062949 (190h34m5.20314308s) [noise=true , address=77.90.31.11:4559 , piece_id=IXHIHQXDLTSVFOPDCAHGYDEI22RBH77SU2DJFZ52MLWZMTANVADQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772183270 (190h34m5.203263401s) [noise=true , address=104.36.82.22:4543 , piece_id=XTN6FE7SKOCWPE6VQTEIFZFZ6OLPFSJSMJRUNYQUU4DJ7JCZR5UQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772563297 (190h34m5.203643428s) [noise=true , address=104.36.82.16:4306 , piece_id=2547BUZR4R5PGGF7MHB2HIWIX42JLKHM4AMNFTSUGYIM55YYYBHQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:11.953464880 (190h34m6.384545011s) [upload=198245]: Next returned nil segment Aug 16 16:00:11.953482854 (190h34m6.384562985s) [upload=198245]: Waiting for error group managing segments... Aug 16 16:00:12.941409550 (190h34m7.372489691s) [noise=true , address=79.127.220.101:4076 , piece_id=LJU4XJG4OFORQVRTD7BOOI47GREELV5F5BGHNZ3VUBBP3HS73JYQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:13.659968773 (190h34m8.091048904s) [noise=true , address=45.91.116.7:4043 , piece_id=2V7JUZMBVKV6IFF6TSHFUTJLVYC7ZWC3AG2F3RXRYZUGWCVVNO6A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1TkHPinJgbiX4PmRfNH7wRxtDZ9rASSvbZtLxZuxbF3UaoSqFD, address:45.91.116.7:4043): protocol: storage node overloaded, request limit: 2000 Aug 16 16:00:14.611558252 (190h34m9.042638383s) [noise=true , address=104.36.82.45:4934 , piece_id=6AMSA3INP3L3ADURWGJDGXTFOFMQ5VSORF3KNGO35ZSPQNYMSP2Q, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:20.670138101 (190h34m15.101218242s) [noise=true , address=66.23.193.175:4557 , piece_id=RDX5DR364JZMUSTWW4HA4P55RNDONJXR676CFUMOG5TO5SJ7JQQA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:22.111492140 (190h34m16.542572311s) [noise=true , address=45.91.116.5:4315 , piece_id=DYNZCXOWKSP6KPULJLUTQHS6JPGPX7WVCESHEH6TTQQIXZM3QZZA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:24.242804738 (190h34m18.673884869s) [noise=true , address=77.90.31.18:4837 , piece_id=5TSI4MM4SHIYCL3PCD3RHYCJR4FXPM7VVVXQNU7ROAZNWHEON4DQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:43.969823882 (190h34m38.400904103s) [noise=true , address=77.90.31.14:4695 , piece_id=ZE67SDK22KFKKAONRYDWRRMG266Q24DHKNVLFNE3NTZDI6XILQSQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:01:09.606812307 (190h35m4.037892438s) [noise=true , address=104.36.82.5:4842 , piece_id=DXZQKVGVMH7TGKVZKRFLI7LTEJY7OBQX6GKRAM4T33CZVRJNGX3A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn, address:104.36.82.5:4842): protocol: rpc: dial tcp 104.36.82.5:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606854288 (190h35m4.037934419s) [noise=true , address=104.36.82.58:4271 , piece_id=M5B7V2YGDPVNZL73XKJH2YKBAGZR7LCKRLCKQGIBMABSRLAHMSWA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1, address:104.36.82.58:4271): protocol: rpc: dial tcp 104.36.82.58:4271: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606884385 (190h35m4.037964516s) [noise=true , address=104.36.82.60:4353 , piece_id=R6OTTIHWGZE2KAS25Z6YCTCWL2RAQKZGTDFI66Y4JYPSK72I7TTA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12BmzVLKbkqPMB7bJHGzFbgRRLu3TLg6TdTBCvYybsvz2yLpBbQ, address:104.36.82.60:4353): protocol: rpc: dial tcp 104.36.82.60:4353: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606917428 (190h35m4.037997559s) [noise=true , address=104.36.82.23:4457 , piece_id=DCL3MYX7AVNV2V3KD5DHFDYYLY3AT6OXE3FEHOGIJB4WLCHQ5KOQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12mpuVB9gpUQ6dNLaMAVaDkgctVSetSbPRxhxU49WhAzycwwNhs, address:104.36.82.23:4457): protocol: write tcp 188.166.225.170:39772->104.36.82.23:4457: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606949459 (190h35m4.03802959s) [noise=true , address=104.36.82.36:4629 , piece_id=K3L4U3YD5N6WDP35N2QKPZ3KLNSCR5R4QWEZS4DMQO4YXOBYXCYQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1uxMhTiH446Ku7h1PutqVh7fSCzwKHR7oypYBzGWELLNP7ojf6, address:104.36.82.36:4629): protocol: write tcp 188.166.225.170:59572->104.36.82.36:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607050022 (190h35m4.038130153s) [noise=true , address=104.36.82.3:4799 , piece_id=LIEZASMYJIYFGFSUGAYBQNS5OY6UPGJ7KWCTBWVMLZ5YMRPGEMTQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:104.36.82.3:4799): protocol: write tcp 188.166.225.170:37514->104.36.82.3:4799: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607085500 (190h35m4.038165631s) [noise=true , address=77.90.31.19:4867 , piece_id=WFPEU7FO65IMB3JAFZ6P3IPIEEOXAW6QDPGK7VRXFLWTSEQMC52A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1gczGyJMCtqUfaac5oy7o4CTwypVF8AZjrAAjoJweU3kBFbuXw, address:77.90.31.19:4867): protocol: write tcp 188.166.225.170:59224->77.90.31.19:4867: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607125927 (190h35m4.038206059s) [noise=true , address=104.36.82.44:4888 , piece_id=TPTSTGF3IAALD6BRRPENCRWLDCWNBEUFB5T6P64Q733SPGL6YOKA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:129FkuTqtaF3DmzHFvcCuJRvaubQV6HbaV1CnFtoY95QHH6NcME, address:104.36.82.44:4888): protocol: rpc: dial tcp 104.36.82.44:4888: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607138701 (190h35m4.038218832s) [noise=true , address=77.90.31.11:4559 , piece_id=IXHIHQXDLTSVFOPDCAHGYDEI22RBH77SU2DJFZ52MLWZMTANVADQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection; write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection Aug 16 16:01:09.607166785 (190h35m4.038246916s) [noise=true , address=104.36.82.22:4543 , piece_id=XTN6FE7SKOCWPE6VQTEIFZFZ6OLPFSJSMJRUNYQUU4DJ7JCZR5UQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:19iUYyo8cGDLGDnvdNo2Qfj1K6tp6EJWF5U6rXD4gJ7Hv7Gsch, address:104.36.82.22:4543): protocol: rpc: dial tcp 104.36.82.22:4543: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607185561 (190h35m4.038265692s) [noise=true , address=104.36.82.16:4306 , piece_id=2547BUZR4R5PGGF7MHB2HIWIX42JLKHM4AMNFTSUGYIM55YYYBHQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1mcaP7H2gd9gWpbcPK79gvemz5L6k5UgbwQ5t6aaTU7nXut1p7, address:104.36.82.16:4306): protocol: rpc: dial tcp 104.36.82.16:4306: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607211200 (190h35m4.038291332s) [seg_pos={0 0}, upload=198245]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 16 16:01:09.843745933 (190h35m4.274826064s) [upload=198245]: Done uploading segments... Aug 16 16:01:09.843772213 (190h35m4.274852344s) [upload=198245]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 6.167126799s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 11 out of 60 is failed (threshold=3) [XXXXXXXX.......X............................X..............X] duration: 6.189782448s FAIL (2025-08-16 16:42:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 16 16:42:16.872285782 (191h16m11.303365913s) [upload=198966]: Starting upload Aug 16 16:42:16.872333563 (191h16m11.303413694s) [upload=198966]: Done starting upload Aug 16 16:42:16.872444025 (191h16m11.303524156s) [upload=198966]: Uploading segments... Aug 16 16:42:16.872541381 (191h16m11.303621512s) [upload=198966]: Got next segment. Inline: false Aug 16 16:42:16.872586317 (191h16m11.303666449s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:42:16.872590746 (191h16m11.303670877s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:42:17.425124264 (191h16m11.856204395s) [seg_pos={0 0}, upload=198966]: Begin upload segment... Aug 16 16:42:17.425172086 (191h16m11.856252217s) [seg_pos={0 0}, upload=198966]: Done begin upload segment. Aug 16 16:42:17.425197014 (191h16m11.856277145s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425203726 (191h16m11.856283857s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425237861 (191h16m11.856317992s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425264162 (191h16m11.856344283s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425575047 (191h16m11.856655168s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425594364 (191h16m11.856674495s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.425866454 (191h16m11.856946585s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426139136 (191h16m11.857219257s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.426399364 (191h16m11.857479496s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427268097 (191h16m11.858348218s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427523435 (191h16m11.858603566s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.427772021 (191h16m11.858852152s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428008213 (191h16m11.859088344s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428234576 (191h16m11.859314697s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.428513469 (191h16m11.859593601s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.429315073 (191h16m11.860395214s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.430699312 (191h16m11.861779443s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:17.432678890 (191h16m11.863759011s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Uploading piece... Aug 16 16:42:18.538182251 (191h16m12.969262382s) [upload=198966]: Next returned nil segment Aug 16 16:42:18.538198913 (191h16m12.969279044s) [upload=198966]: Waiting for error group managing segments... Aug 16 16:42:18.819869036 (191h16m13.250949157s) [noise=true , address=66.23.193.175:4557 , piece_id=5GC7J36MPILKKQYX7AWXKYWBVMENOMG2KHEKUBOSETPEMIZPUYBA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.312426918 (191h16m13.743507049s) [noise=true , address=79.127.220.101:4071 , piece_id=QBTVWJHWJ4HXLFF3BGXQHXZ4FMHMCAR7A4SJCOO2F2FXF6E5NKMQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.379261348 (191h16m13.810341479s) [noise=true , address=45.91.116.6:4982 , piece_id=USHJD7ZJEUR35NBPTEMAUGSSRQ5ZUWU4UWX4PUWGYPSEDCSHSH4A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.389869235 (191h16m13.820949366s) [noise=true , address=77.90.31.19:4876 , piece_id=63UNIBZNZBKMDFHUMIAF5V5I77HXGX6UNDSXJKPDRKW7CEIXJYFA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.393817941 (191h16m13.824898072s) [noise=true , address=77.90.31.21:4883 , piece_id=WMIMWIFV4PBI6LCLL5ZCZGOEEMI6LMBFASQFUPYCUKXV5DWGWR2A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.411086712 (191h16m13.842166843s) [noise=true , address=77.90.31.10:4526 , piece_id=JY32PBGS4PUCHENMHXRMNICBL4XFQVPL52ZWSJ3D427PHXUIT5YA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.422110104 (191h16m13.853190225s) [noise=true , address=104.36.82.8:4979 , piece_id=LFRDYDNAJC4LCZXTFOHFBL2BOEO6P37JFP3XHOIJJRJR5IZAG2IQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.437138790 (191h16m13.868218921s) [noise=true , address=45.91.116.8:4072 , piece_id=724337FVNPCS2XK3OOFUO2UF6VUHNCO5BKT7GQOKYETZJKMSGQKQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.440196149 (191h16m13.87127627s) [noise=true , address=45.91.116.7:4041 , piece_id=5YOFMHHJADLDVNIPYVFQHLOGQME2ETD6GTRL4ETNJQGZIUBBCIPQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.472446607 (191h16m13.903526738s) [noise=true , address=104.36.82.4:4836 , piece_id=QG6BJPSP7JHJZJUEU2XDLYX2H3VNRBINGYCLWJ3K3GTOVNROK77A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.534216617 (191h16m13.965296748s) [noise=true , address=104.36.82.36:4625 , piece_id=ZWP3JV22W7DLCST2JMJPDJWAKL4IRIQKRNOFCTNSCL3K7HULU3RQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.547051065 (191h16m13.978131196s) [noise=true , address=104.36.82.2:4740 , piece_id=U4UMZICGBVMMMPKYEQON5QRGL26D4HXVSWHQHP6ZX4UILIAK2H2Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.549328613 (191h16m13.980408734s) [noise=true , address=77.90.31.13:4652 , piece_id=5C5UAESIUJVBX57QVEYGCKKJEJZKA6NGUTO7FLJBQPDGYPIS6PPA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563132655 (191h16m13.994212786s) [noise=true , address=104.36.82.23:4454 , piece_id=YZ654RNLNJFE5QAOPMYMGJJKSWU6RDRN6LJP2HBIDAXMT2SIZD4Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: <nil> Aug 16 16:42:19.563159145 (191h16m13.994239277s) [seg_pos={0 0}, upload=198966]: Segment reached optimal threshold of 14 pieces. Aug 16 16:42:19.563278804 (191h16m13.994358935s) [noise=true , address=104.36.82.58:4276 , piece_id=JL4FECE7S5ASMQYYN4QG3T23L33WJRB45SYCIMWPPWUEJFXNR6KA, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563335653 (191h16m13.994415784s) [noise=true , address=45.91.116.14:4207 , piece_id=XAOVXDLYLJV4G4TEYA6VBUGCYL7C7CMZKUEYZ75TD7KPXQ5HE64A, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563407671 (191h16m13.994487802s) [noise=true , address=104.36.82.59:4311 , piece_id=JNXLALHDPRPG2E53ZZDH5HIR7RWTTVB4N2HA6W3TUOBT6MLC6FXQ, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563455042 (191h16m13.994535223s) [noise=true , address=77.90.31.11:4575 , piece_id=N4SQHAI2IHXYQ3KJKXVSMTDQTSI5Y6NSW2ZRAVYNZHXCXBMXK77Q, seg_pos={0 0}, upload=198966]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Aug 16 16:42:19.563485821 (191h16m13.994565942s) [seg_pos={0 0}, upload=198966]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 16:42:19.563543791 (191h16m13.994623922s) [upload=198966]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 16:42:19.563547639 (191h16m13.99462777s) [upload=198966]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:42:21.134499561 (191h16m15.565579702s) [upload=198966]: Done uploading segments... Aug 16 16:42:21.134524349 (191h16m15.56560449s) [upload=198966]: Upload finished. err: <nil> Duration: 6.189810862s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 6.126229678s FAIL (2025-08-16 16:42:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== . Aug 16 15:59:57.963918156 (190h33m52.394998287s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964029729 (190h33m52.39510986s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964451335 (190h33m52.395531467s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964727965 (190h33m52.395808086s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964797899 (190h33m52.39587802s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965042367 (190h33m52.396122498s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965090028 (190h33m52.396170159s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965305831 (190h33m52.396385962s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965343212 (190h33m52.396423344s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965548745 (190h33m52.396628876s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965569796 (190h33m52.396649927s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965765831 (190h33m52.396845962s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966003606 (190h33m52.397083737s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966271599 (190h33m52.39735173s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:59.736319114 (190h33m54.167399245s) [upload=198240]: Next returned nil segment Aug 16 15:59:59.736337630 (190h33m54.167417761s) [upload=198240]: Waiting for error group managing segments... Aug 16 16:00:00.028564434 (190h33m54.459644565s) [noise=true , address=104.36.82.6:4874 , piece_id=FI3EZZH5G2O42GFDQGX6ZRLX2KFM55UTNMCJY7AGYS2LPH4WCO3Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.092359868 (190h33m54.523439999s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.158899656 (190h33m54.589979787s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.274239525 (190h33m54.705319646s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:08.612621871 (190h34m3.043702012s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:25.194912489 (190h34m19.62599263s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:27.035229138 (190h34m21.466309269s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035398122 (190h34m21.466478263s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035640335 (190h34m21.466720496s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection Aug 16 16:00:27.035680382 (190h34m21.466760513s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection Aug 16 16:00:27.035716621 (190h34m21.466796752s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection Aug 16 16:00:27.035748352 (190h34m21.466828483s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: rpc: dial tcp 104.36.82.16:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035901174 (190h34m21.466981305s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 188.166.225.170:33266->104.36.82.11:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035939287 (190h34m21.467019418s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection Aug 16 16:00:27.035966249 (190h34m21.46704638s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection Aug 16 16:00:27.036250663 (190h34m21.467330794s) [noise=true , address=104.36.82.8:4970 , piece_id=XIPQ7G4S7ETV46ME3HDJTUUXDTNGWS2ZX75VO57NV4R35OGLFZOA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: context deadline exceeded; context deadline exceeded Aug 16 16:00:27.036322741 (190h34m21.467402872s) [seg_pos={0 0}, upload=198240]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 16 16:00:27.438563272 (190h34m21.869643404s) [upload=198240]: Done uploading segments... Aug 16 16:00:27.438589974 (190h34m21.869670105s) [upload=198240]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.126250649s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........X.....................................X.....X.....] duration: 2.666479567s FAIL (2025-08-16 16:42:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.666509355s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 11 out of 60 is failed (threshold=3) [XXXXXXXXX.......X............................X..............] duration: 8.084756501s FAIL (2025-08-16 16:42:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 16:29:58.897297867 (191h3m53.328377988s) [upload=198750]: Starting upload Aug 16 16:29:58.897347813 (191h3m53.328427934s) [upload=198750]: Done starting upload Aug 16 16:29:58.897431863 (191h3m53.328511995s) [upload=198750]: Uploading segments... Aug 16 16:29:58.897506477 (191h3m53.328586598s) [upload=198750]: Got next segment. Inline: false Aug 16 16:29:58.897548347 (191h3m53.328628478s) [upload=198750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:29:58.897552896 (191h3m53.328633027s) [upload=198750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:29:59.111406047 (191h3m53.542486178s) [upload=198750]: Next segment err: metaclient: Too Many Requests Aug 16 16:29:59.111428409 (191h3m53.54250854s) [upload=198750]: Done uploading segments... Aug 16 16:29:59.111442216 (191h3m53.542522347s) [upload=198750]: Upload finished. err: metaclient: Too Many Requests Duration: 8.084978276s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........X.....................................X.....X....] duration: 4.485506969s FAIL (2025-08-16 16:41:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 4.485524923s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 12 out of 60 is failed (threshold=3) [XXXXXXXXXX.......X............................X.............] duration: 7.293623612s FAIL (2025-08-16 16:41:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 16:29:58.897297867 (191h3m53.328377988s) [upload=198750]: Starting upload Aug 16 16:29:58.897347813 (191h3m53.328427934s) [upload=198750]: Done starting upload Aug 16 16:29:58.897431863 (191h3m53.328511995s) [upload=198750]: Uploading segments... Aug 16 16:29:58.897506477 (191h3m53.328586598s) [upload=198750]: Got next segment. Inline: false Aug 16 16:29:58.897548347 (191h3m53.328628478s) [upload=198750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:29:58.897552896 (191h3m53.328633027s) [upload=198750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 16:29:59.111406047 (191h3m53.542486178s) [upload=198750]: Next segment err: metaclient: Too Many Requests Aug 16 16:29:59.111428409 (191h3m53.54250854s) [upload=198750]: Done uploading segments... Aug 16 16:29:59.111442216 (191h3m53.542522347s) [upload=198750]: Upload finished. err: metaclient: Too Many Requests Duration: 7.293649641s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 6.383767127s FAIL (2025-08-16 16:41:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== upload=198245]: Uploading piece... Aug 16 16:00:10.770806034 (190h34m5.201886165s) [noise=true , address=104.36.82.58:4271 , piece_id=M5B7V2YGDPVNZL73XKJH2YKBAGZR7LCKRLCKQGIBMABSRLAHMSWA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.770975228 (190h34m5.202055359s) [noise=true , address=104.36.82.60:4353 , piece_id=R6OTTIHWGZE2KAS25Z6YCTCWL2RAQKZGTDFI66Y4JYPSK72I7TTA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771077693 (190h34m5.202157825s) [noise=true , address=77.90.31.14:4695 , piece_id=ZE67SDK22KFKKAONRYDWRRMG266Q24DHKNVLFNE3NTZDI6XILQSQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771274441 (190h34m5.202354572s) [noise=true , address=104.36.82.45:4934 , piece_id=6AMSA3INP3L3ADURWGJDGXTFOFMQ5VSORF3KNGO35ZSPQNYMSP2Q, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771309197 (190h34m5.202389328s) [noise=true , address=104.36.82.44:4888 , piece_id=TPTSTGF3IAALD6BRRPENCRWLDCWNBEUFB5T6P64Q733SPGL6YOKA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771570006 (190h34m5.202650137s) [noise=true , address=104.36.82.5:4842 , piece_id=DXZQKVGVMH7TGKVZKRFLI7LTEJY7OBQX6GKRAM4T33CZVRJNGX3A, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771594673 (190h34m5.202674804s) [noise=true , address=79.127.220.101:4076 , piece_id=LJU4XJG4OFORQVRTD7BOOI47GREELV5F5BGHNZ3VUBBP3HS73JYQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771829703 (190h34m5.202909834s) [noise=true , address=104.36.82.36:4629 , piece_id=K3L4U3YD5N6WDP35N2QKPZ3KLNSCR5R4QWEZS4DMQO4YXOBYXCYQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771875119 (190h34m5.20295527s) [noise=true , address=45.91.116.7:4043 , piece_id=2V7JUZMBVKV6IFF6TSHFUTJLVYC7ZWC3AG2F3RXRYZUGWCVVNO6A, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772062949 (190h34m5.20314308s) [noise=true , address=77.90.31.11:4559 , piece_id=IXHIHQXDLTSVFOPDCAHGYDEI22RBH77SU2DJFZ52MLWZMTANVADQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772183270 (190h34m5.203263401s) [noise=true , address=104.36.82.22:4543 , piece_id=XTN6FE7SKOCWPE6VQTEIFZFZ6OLPFSJSMJRUNYQUU4DJ7JCZR5UQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772563297 (190h34m5.203643428s) [noise=true , address=104.36.82.16:4306 , piece_id=2547BUZR4R5PGGF7MHB2HIWIX42JLKHM4AMNFTSUGYIM55YYYBHQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:11.953464880 (190h34m6.384545011s) [upload=198245]: Next returned nil segment Aug 16 16:00:11.953482854 (190h34m6.384562985s) [upload=198245]: Waiting for error group managing segments... Aug 16 16:00:12.941409550 (190h34m7.372489691s) [noise=true , address=79.127.220.101:4076 , piece_id=LJU4XJG4OFORQVRTD7BOOI47GREELV5F5BGHNZ3VUBBP3HS73JYQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:13.659968773 (190h34m8.091048904s) [noise=true , address=45.91.116.7:4043 , piece_id=2V7JUZMBVKV6IFF6TSHFUTJLVYC7ZWC3AG2F3RXRYZUGWCVVNO6A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1TkHPinJgbiX4PmRfNH7wRxtDZ9rASSvbZtLxZuxbF3UaoSqFD, address:45.91.116.7:4043): protocol: storage node overloaded, request limit: 2000 Aug 16 16:00:14.611558252 (190h34m9.042638383s) [noise=true , address=104.36.82.45:4934 , piece_id=6AMSA3INP3L3ADURWGJDGXTFOFMQ5VSORF3KNGO35ZSPQNYMSP2Q, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:20.670138101 (190h34m15.101218242s) [noise=true , address=66.23.193.175:4557 , piece_id=RDX5DR364JZMUSTWW4HA4P55RNDONJXR676CFUMOG5TO5SJ7JQQA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:22.111492140 (190h34m16.542572311s) [noise=true , address=45.91.116.5:4315 , piece_id=DYNZCXOWKSP6KPULJLUTQHS6JPGPX7WVCESHEH6TTQQIXZM3QZZA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:24.242804738 (190h34m18.673884869s) [noise=true , address=77.90.31.18:4837 , piece_id=5TSI4MM4SHIYCL3PCD3RHYCJR4FXPM7VVVXQNU7ROAZNWHEON4DQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:43.969823882 (190h34m38.400904103s) [noise=true , address=77.90.31.14:4695 , piece_id=ZE67SDK22KFKKAONRYDWRRMG266Q24DHKNVLFNE3NTZDI6XILQSQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:01:09.606812307 (190h35m4.037892438s) [noise=true , address=104.36.82.5:4842 , piece_id=DXZQKVGVMH7TGKVZKRFLI7LTEJY7OBQX6GKRAM4T33CZVRJNGX3A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn, address:104.36.82.5:4842): protocol: rpc: dial tcp 104.36.82.5:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606854288 (190h35m4.037934419s) [noise=true , address=104.36.82.58:4271 , piece_id=M5B7V2YGDPVNZL73XKJH2YKBAGZR7LCKRLCKQGIBMABSRLAHMSWA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1, address:104.36.82.58:4271): protocol: rpc: dial tcp 104.36.82.58:4271: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606884385 (190h35m4.037964516s) [noise=true , address=104.36.82.60:4353 , piece_id=R6OTTIHWGZE2KAS25Z6YCTCWL2RAQKZGTDFI66Y4JYPSK72I7TTA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12BmzVLKbkqPMB7bJHGzFbgRRLu3TLg6TdTBCvYybsvz2yLpBbQ, address:104.36.82.60:4353): protocol: rpc: dial tcp 104.36.82.60:4353: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606917428 (190h35m4.037997559s) [noise=true , address=104.36.82.23:4457 , piece_id=DCL3MYX7AVNV2V3KD5DHFDYYLY3AT6OXE3FEHOGIJB4WLCHQ5KOQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12mpuVB9gpUQ6dNLaMAVaDkgctVSetSbPRxhxU49WhAzycwwNhs, address:104.36.82.23:4457): protocol: write tcp 188.166.225.170:39772->104.36.82.23:4457: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606949459 (190h35m4.03802959s) [noise=true , address=104.36.82.36:4629 , piece_id=K3L4U3YD5N6WDP35N2QKPZ3KLNSCR5R4QWEZS4DMQO4YXOBYXCYQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1uxMhTiH446Ku7h1PutqVh7fSCzwKHR7oypYBzGWELLNP7ojf6, address:104.36.82.36:4629): protocol: write tcp 188.166.225.170:59572->104.36.82.36:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607050022 (190h35m4.038130153s) [noise=true , address=104.36.82.3:4799 , piece_id=LIEZASMYJIYFGFSUGAYBQNS5OY6UPGJ7KWCTBWVMLZ5YMRPGEMTQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:104.36.82.3:4799): protocol: write tcp 188.166.225.170:37514->104.36.82.3:4799: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607085500 (190h35m4.038165631s) [noise=true , address=77.90.31.19:4867 , piece_id=WFPEU7FO65IMB3JAFZ6P3IPIEEOXAW6QDPGK7VRXFLWTSEQMC52A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1gczGyJMCtqUfaac5oy7o4CTwypVF8AZjrAAjoJweU3kBFbuXw, address:77.90.31.19:4867): protocol: write tcp 188.166.225.170:59224->77.90.31.19:4867: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607125927 (190h35m4.038206059s) [noise=true , address=104.36.82.44:4888 , piece_id=TPTSTGF3IAALD6BRRPENCRWLDCWNBEUFB5T6P64Q733SPGL6YOKA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:129FkuTqtaF3DmzHFvcCuJRvaubQV6HbaV1CnFtoY95QHH6NcME, address:104.36.82.44:4888): protocol: rpc: dial tcp 104.36.82.44:4888: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607138701 (190h35m4.038218832s) [noise=true , address=77.90.31.11:4559 , piece_id=IXHIHQXDLTSVFOPDCAHGYDEI22RBH77SU2DJFZ52MLWZMTANVADQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection; write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection Aug 16 16:01:09.607166785 (190h35m4.038246916s) [noise=true , address=104.36.82.22:4543 , piece_id=XTN6FE7SKOCWPE6VQTEIFZFZ6OLPFSJSMJRUNYQUU4DJ7JCZR5UQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:19iUYyo8cGDLGDnvdNo2Qfj1K6tp6EJWF5U6rXD4gJ7Hv7Gsch, address:104.36.82.22:4543): protocol: rpc: dial tcp 104.36.82.22:4543: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607185561 (190h35m4.038265692s) [noise=true , address=104.36.82.16:4306 , piece_id=2547BUZR4R5PGGF7MHB2HIWIX42JLKHM4AMNFTSUGYIM55YYYBHQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1mcaP7H2gd9gWpbcPK79gvemz5L6k5UgbwQ5t6aaTU7nXut1p7, address:104.36.82.16:4306): protocol: rpc: dial tcp 104.36.82.16:4306: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607211200 (190h35m4.038291332s) [seg_pos={0 0}, upload=198245]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 16 16:01:09.843745933 (190h35m4.274826064s) [upload=198245]: Done uploading segments... Aug 16 16:01:09.843772213 (190h35m4.274852344s) [upload=198245]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 6.383796974s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 5.699727705s FAIL (2025-08-16 16:41:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== . Aug 16 15:59:57.963918156 (190h33m52.394998287s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964029729 (190h33m52.39510986s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964451335 (190h33m52.395531467s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964727965 (190h33m52.395808086s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964797899 (190h33m52.39587802s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965042367 (190h33m52.396122498s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965090028 (190h33m52.396170159s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965305831 (190h33m52.396385962s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965343212 (190h33m52.396423344s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965548745 (190h33m52.396628876s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965569796 (190h33m52.396649927s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965765831 (190h33m52.396845962s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966003606 (190h33m52.397083737s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966271599 (190h33m52.39735173s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:59.736319114 (190h33m54.167399245s) [upload=198240]: Next returned nil segment Aug 16 15:59:59.736337630 (190h33m54.167417761s) [upload=198240]: Waiting for error group managing segments... Aug 16 16:00:00.028564434 (190h33m54.459644565s) [noise=true , address=104.36.82.6:4874 , piece_id=FI3EZZH5G2O42GFDQGX6ZRLX2KFM55UTNMCJY7AGYS2LPH4WCO3Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.092359868 (190h33m54.523439999s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.158899656 (190h33m54.589979787s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.274239525 (190h33m54.705319646s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:08.612621871 (190h34m3.043702012s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:25.194912489 (190h34m19.62599263s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:27.035229138 (190h34m21.466309269s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035398122 (190h34m21.466478263s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035640335 (190h34m21.466720496s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection Aug 16 16:00:27.035680382 (190h34m21.466760513s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection Aug 16 16:00:27.035716621 (190h34m21.466796752s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection Aug 16 16:00:27.035748352 (190h34m21.466828483s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: rpc: dial tcp 104.36.82.16:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035901174 (190h34m21.466981305s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 188.166.225.170:33266->104.36.82.11:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035939287 (190h34m21.467019418s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection Aug 16 16:00:27.035966249 (190h34m21.46704638s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection Aug 16 16:00:27.036250663 (190h34m21.467330794s) [noise=true , address=104.36.82.8:4970 , piece_id=XIPQ7G4S7ETV46ME3HDJTUUXDTNGWS2ZX75VO57NV4R35OGLFZOA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: context deadline exceeded; context deadline exceeded Aug 16 16:00:27.036322741 (190h34m21.467402872s) [seg_pos={0 0}, upload=198240]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 16 16:00:27.438563272 (190h34m21.869643404s) [upload=198240]: Done uploading segments... Aug 16 16:00:27.438589974 (190h34m21.869670105s) [upload=198240]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.699749617s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............X.....................................X.....X...] duration: 2.579374465s FAIL (2025-08-16 16:40:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b904a7bddc1e5982c1dc71fca8b13370aa076cda4254f7a7ebb7921474868aae != before 5e378825b1eab5f398f07314b4ecb6868c630ee8ca0697ac953a548cb8d4f6f1 ======LOGS====== Aug 16 16:38:08.855836099 (191h12m3.28691623s) [upload=198889]: Starting upload Aug 16 16:38:08.855888369 (191h12m3.28696849s) [upload=198889]: Done starting upload Aug 16 16:38:08.855935159 (191h12m3.28701528s) [upload=198889]: Uploading segments... Aug 16 16:38:08.855966158 (191h12m3.287046289s) [upload=198889]: Got next segment. Inline: true Aug 16 16:38:08.855970216 (191h12m3.287050347s) [upload=198889]: Waiting for error group managing segments... Aug 16 16:38:08.856001275 (191h12m3.287081406s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:38:08.856004982 (191h12m3.287085113s) [upload=198889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:38:08.856007457 (191h12m3.287087588s) [upload=198889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:38:16.879147132 (191h12m11.310227263s) [upload=198889]: Done uploading segments... Aug 16 16:38:16.879164886 (191h12m11.310245017s) [upload=198889]: Upload finished. err: <nil> Duration:* 2.57940358s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 5.804474947s FAIL (2025-08-16 16:40:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== . Aug 16 15:59:57.963918156 (190h33m52.394998287s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964029729 (190h33m52.39510986s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964451335 (190h33m52.395531467s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964727965 (190h33m52.395808086s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.964797899 (190h33m52.39587802s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965042367 (190h33m52.396122498s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965090028 (190h33m52.396170159s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965305831 (190h33m52.396385962s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965343212 (190h33m52.396423344s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965548745 (190h33m52.396628876s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965569796 (190h33m52.396649927s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.965765831 (190h33m52.396845962s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966003606 (190h33m52.397083737s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:57.966271599 (190h33m52.39735173s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Uploading piece... Aug 16 15:59:59.736319114 (190h33m54.167399245s) [upload=198240]: Next returned nil segment Aug 16 15:59:59.736337630 (190h33m54.167417761s) [upload=198240]: Waiting for error group managing segments... Aug 16 16:00:00.028564434 (190h33m54.459644565s) [noise=true , address=104.36.82.6:4874 , piece_id=FI3EZZH5G2O42GFDQGX6ZRLX2KFM55UTNMCJY7AGYS2LPH4WCO3Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.092359868 (190h33m54.523439999s) [noise=true , address=104.36.82.7:4912 , piece_id=QG2PMXGDHI7YDPOWL5NBXX3C2N2EHVY2CL6X3QICJZABHVLJKI3A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.158899656 (190h33m54.589979787s) [noise=true , address=104.36.82.10:4079 , piece_id=B3RYSKZKQPUPL4ZHJ6IAOKGF7XPNJMJPWKEGQ2P3HC2WLCGUJ2XA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:00.274239525 (190h33m54.705319646s) [noise=true , address=104.36.82.15:4258 , piece_id=GJJ3FE23W7G5VYXQ7VFF4F6H4RECYP6I4QTY5UYFMPA2XF6APABA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:08.612621871 (190h34m3.043702012s) [noise=true , address=45.91.116.6:4974 , piece_id=KXQZPRG5V5AJF56XD7YRTHAUOP7RCOACV3TIYDA7R7DACD3LID2A, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:25.194912489 (190h34m19.62599263s) [noise=true , address=45.91.116.9:4263 , piece_id=2TC6SI42AFPDXKYWKKEFWDIJLVLAH2TAQ6SKJYDYGBEEYYTQ3IYQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: <nil> Aug 16 16:00:27.035229138 (190h34m21.466309269s) [noise=true , address=45.91.116.12:4227 , piece_id=H7VBILB3XMAMH3XOVKS57O7J6ZKMAG5LEUB25MHXZPSGUX4ZALUQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035398122 (190h34m21.466478263s) [noise=true , address=45.91.116.4:4939 , piece_id=LK6XVI4CHDVZGHGL4QZIZABS2XCWGRI6DEJDKY7CC5PVEB4V5KAA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 16:00:27.035640335 (190h34m21.466720496s) [noise=true , address=45.91.116.5:4297 , piece_id=AZCIMUM2ATU333E5X224BJEGA4ZDGTXNSWLQ7EWQ2XVRUO3SJXQQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46792->45.91.116.5:4297: use of closed network connection Aug 16 16:00:27.035680382 (190h34m21.466760513s) [noise=true , address=45.91.116.14:4190 , piece_id=JJGVJBLIICUJDJD6R7AQY3DL3SAYONGBIY2NRGLJOFWW6WHFKM6Q, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49854->45.91.116.14:4190: use of closed network connection Aug 16 16:00:27.035716621 (190h34m21.466796752s) [noise=true , address=45.91.116.4:4940 , piece_id=JVCUNP6OX3DFZGTJY6FNCJE55FRR3BT6D3UONINJF4Y6GCTJ3FXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56748->45.91.116.4:4940: use of closed network connection Aug 16 16:00:27.035748352 (190h34m21.466828483s) [noise=true , address=104.36.82.16:4293 , piece_id=DM6MQ7NBGY3QEPF5PKLNOHH2SCKKYBI25TF6PAR4QRX2BPHDVLXQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: rpc: dial tcp 104.36.82.16:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035901174 (190h34m21.466981305s) [noise=true , address=104.36.82.11:4115 , piece_id=OMI5C3GY4N6YCYVC75KM5ZBYCFPUE3Y4YYSK5PEBZVDIAC4V5QPA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 188.166.225.170:33266->104.36.82.11:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:00:27.035939287 (190h34m21.467019418s) [noise=true , address=45.91.116.7:4011 , piece_id=UXAIDQXZAV7LTIMWELENQWGLBUMLOMQN5WK32XBAGSGPQTIO4MPQ, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43028->45.91.116.7:4011: use of closed network connection Aug 16 16:00:27.035966249 (190h34m21.46704638s) [noise=true , address=77.90.31.14:4663 , piece_id=AQ4UKW62AAVZ3WMTR7GCBNPCLF2U3F2ZUO6RH7VPI27EKSTM4UGA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55318->77.90.31.14:4663: use of closed network connection Aug 16 16:00:27.036250663 (190h34m21.467330794s) [noise=true , address=104.36.82.8:4970 , piece_id=XIPQ7G4S7ETV46ME3HDJTUUXDTNGWS2ZX75VO57NV4R35OGLFZOA, seg_pos={0 0}, upload=198240]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: context deadline exceeded; context deadline exceeded Aug 16 16:00:27.036322741 (190h34m21.467402872s) [seg_pos={0 0}, upload=198240]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 16 16:00:27.438563272 (190h34m21.869643404s) [upload=198240]: Done uploading segments... Aug 16 16:00:27.438589974 (190h34m21.869670105s) [upload=198240]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.804499555s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [XXXXX.......................................................] duration: 6.151200407s FAIL (2025-08-16 16:40:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== upload=198245]: Uploading piece... Aug 16 16:00:10.770806034 (190h34m5.201886165s) [noise=true , address=104.36.82.58:4271 , piece_id=M5B7V2YGDPVNZL73XKJH2YKBAGZR7LCKRLCKQGIBMABSRLAHMSWA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.770975228 (190h34m5.202055359s) [noise=true , address=104.36.82.60:4353 , piece_id=R6OTTIHWGZE2KAS25Z6YCTCWL2RAQKZGTDFI66Y4JYPSK72I7TTA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771077693 (190h34m5.202157825s) [noise=true , address=77.90.31.14:4695 , piece_id=ZE67SDK22KFKKAONRYDWRRMG266Q24DHKNVLFNE3NTZDI6XILQSQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771274441 (190h34m5.202354572s) [noise=true , address=104.36.82.45:4934 , piece_id=6AMSA3INP3L3ADURWGJDGXTFOFMQ5VSORF3KNGO35ZSPQNYMSP2Q, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771309197 (190h34m5.202389328s) [noise=true , address=104.36.82.44:4888 , piece_id=TPTSTGF3IAALD6BRRPENCRWLDCWNBEUFB5T6P64Q733SPGL6YOKA, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771570006 (190h34m5.202650137s) [noise=true , address=104.36.82.5:4842 , piece_id=DXZQKVGVMH7TGKVZKRFLI7LTEJY7OBQX6GKRAM4T33CZVRJNGX3A, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771594673 (190h34m5.202674804s) [noise=true , address=79.127.220.101:4076 , piece_id=LJU4XJG4OFORQVRTD7BOOI47GREELV5F5BGHNZ3VUBBP3HS73JYQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771829703 (190h34m5.202909834s) [noise=true , address=104.36.82.36:4629 , piece_id=K3L4U3YD5N6WDP35N2QKPZ3KLNSCR5R4QWEZS4DMQO4YXOBYXCYQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.771875119 (190h34m5.20295527s) [noise=true , address=45.91.116.7:4043 , piece_id=2V7JUZMBVKV6IFF6TSHFUTJLVYC7ZWC3AG2F3RXRYZUGWCVVNO6A, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772062949 (190h34m5.20314308s) [noise=true , address=77.90.31.11:4559 , piece_id=IXHIHQXDLTSVFOPDCAHGYDEI22RBH77SU2DJFZ52MLWZMTANVADQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772183270 (190h34m5.203263401s) [noise=true , address=104.36.82.22:4543 , piece_id=XTN6FE7SKOCWPE6VQTEIFZFZ6OLPFSJSMJRUNYQUU4DJ7JCZR5UQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:10.772563297 (190h34m5.203643428s) [noise=true , address=104.36.82.16:4306 , piece_id=2547BUZR4R5PGGF7MHB2HIWIX42JLKHM4AMNFTSUGYIM55YYYBHQ, seg_pos={0 0}, upload=198245]: Uploading piece... Aug 16 16:00:11.953464880 (190h34m6.384545011s) [upload=198245]: Next returned nil segment Aug 16 16:00:11.953482854 (190h34m6.384562985s) [upload=198245]: Waiting for error group managing segments... Aug 16 16:00:12.941409550 (190h34m7.372489691s) [noise=true , address=79.127.220.101:4076 , piece_id=LJU4XJG4OFORQVRTD7BOOI47GREELV5F5BGHNZ3VUBBP3HS73JYQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:13.659968773 (190h34m8.091048904s) [noise=true , address=45.91.116.7:4043 , piece_id=2V7JUZMBVKV6IFF6TSHFUTJLVYC7ZWC3AG2F3RXRYZUGWCVVNO6A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1TkHPinJgbiX4PmRfNH7wRxtDZ9rASSvbZtLxZuxbF3UaoSqFD, address:45.91.116.7:4043): protocol: storage node overloaded, request limit: 2000 Aug 16 16:00:14.611558252 (190h34m9.042638383s) [noise=true , address=104.36.82.45:4934 , piece_id=6AMSA3INP3L3ADURWGJDGXTFOFMQ5VSORF3KNGO35ZSPQNYMSP2Q, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:20.670138101 (190h34m15.101218242s) [noise=true , address=66.23.193.175:4557 , piece_id=RDX5DR364JZMUSTWW4HA4P55RNDONJXR676CFUMOG5TO5SJ7JQQA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:22.111492140 (190h34m16.542572311s) [noise=true , address=45.91.116.5:4315 , piece_id=DYNZCXOWKSP6KPULJLUTQHS6JPGPX7WVCESHEH6TTQQIXZM3QZZA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:24.242804738 (190h34m18.673884869s) [noise=true , address=77.90.31.18:4837 , piece_id=5TSI4MM4SHIYCL3PCD3RHYCJR4FXPM7VVVXQNU7ROAZNWHEON4DQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:00:43.969823882 (190h34m38.400904103s) [noise=true , address=77.90.31.14:4695 , piece_id=ZE67SDK22KFKKAONRYDWRRMG266Q24DHKNVLFNE3NTZDI6XILQSQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: <nil> Aug 16 16:01:09.606812307 (190h35m4.037892438s) [noise=true , address=104.36.82.5:4842 , piece_id=DXZQKVGVMH7TGKVZKRFLI7LTEJY7OBQX6GKRAM4T33CZVRJNGX3A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn, address:104.36.82.5:4842): protocol: rpc: dial tcp 104.36.82.5:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606854288 (190h35m4.037934419s) [noise=true , address=104.36.82.58:4271 , piece_id=M5B7V2YGDPVNZL73XKJH2YKBAGZR7LCKRLCKQGIBMABSRLAHMSWA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1, address:104.36.82.58:4271): protocol: rpc: dial tcp 104.36.82.58:4271: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606884385 (190h35m4.037964516s) [noise=true , address=104.36.82.60:4353 , piece_id=R6OTTIHWGZE2KAS25Z6YCTCWL2RAQKZGTDFI66Y4JYPSK72I7TTA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12BmzVLKbkqPMB7bJHGzFbgRRLu3TLg6TdTBCvYybsvz2yLpBbQ, address:104.36.82.60:4353): protocol: rpc: dial tcp 104.36.82.60:4353: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606917428 (190h35m4.037997559s) [noise=true , address=104.36.82.23:4457 , piece_id=DCL3MYX7AVNV2V3KD5DHFDYYLY3AT6OXE3FEHOGIJB4WLCHQ5KOQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12mpuVB9gpUQ6dNLaMAVaDkgctVSetSbPRxhxU49WhAzycwwNhs, address:104.36.82.23:4457): protocol: write tcp 188.166.225.170:39772->104.36.82.23:4457: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.606949459 (190h35m4.03802959s) [noise=true , address=104.36.82.36:4629 , piece_id=K3L4U3YD5N6WDP35N2QKPZ3KLNSCR5R4QWEZS4DMQO4YXOBYXCYQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1uxMhTiH446Ku7h1PutqVh7fSCzwKHR7oypYBzGWELLNP7ojf6, address:104.36.82.36:4629): protocol: write tcp 188.166.225.170:59572->104.36.82.36:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607050022 (190h35m4.038130153s) [noise=true , address=104.36.82.3:4799 , piece_id=LIEZASMYJIYFGFSUGAYBQNS5OY6UPGJ7KWCTBWVMLZ5YMRPGEMTQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:104.36.82.3:4799): protocol: write tcp 188.166.225.170:37514->104.36.82.3:4799: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607085500 (190h35m4.038165631s) [noise=true , address=77.90.31.19:4867 , piece_id=WFPEU7FO65IMB3JAFZ6P3IPIEEOXAW6QDPGK7VRXFLWTSEQMC52A, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1gczGyJMCtqUfaac5oy7o4CTwypVF8AZjrAAjoJweU3kBFbuXw, address:77.90.31.19:4867): protocol: write tcp 188.166.225.170:59224->77.90.31.19:4867: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607125927 (190h35m4.038206059s) [noise=true , address=104.36.82.44:4888 , piece_id=TPTSTGF3IAALD6BRRPENCRWLDCWNBEUFB5T6P64Q733SPGL6YOKA, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:129FkuTqtaF3DmzHFvcCuJRvaubQV6HbaV1CnFtoY95QHH6NcME, address:104.36.82.44:4888): protocol: rpc: dial tcp 104.36.82.44:4888: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607138701 (190h35m4.038218832s) [noise=true , address=77.90.31.11:4559 , piece_id=IXHIHQXDLTSVFOPDCAHGYDEI22RBH77SU2DJFZ52MLWZMTANVADQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection; write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36778->77.90.31.11:4559: use of closed network connection Aug 16 16:01:09.607166785 (190h35m4.038246916s) [noise=true , address=104.36.82.22:4543 , piece_id=XTN6FE7SKOCWPE6VQTEIFZFZ6OLPFSJSMJRUNYQUU4DJ7JCZR5UQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:19iUYyo8cGDLGDnvdNo2Qfj1K6tp6EJWF5U6rXD4gJ7Hv7Gsch, address:104.36.82.22:4543): protocol: rpc: dial tcp 104.36.82.22:4543: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607185561 (190h35m4.038265692s) [noise=true , address=104.36.82.16:4306 , piece_id=2547BUZR4R5PGGF7MHB2HIWIX42JLKHM4AMNFTSUGYIM55YYYBHQ, seg_pos={0 0}, upload=198245]: Done uploading piece. err: ecclient: upload failed (node:1mcaP7H2gd9gWpbcPK79gvemz5L6k5UgbwQ5t6aaTU7nXut1p7, address:104.36.82.16:4306): protocol: rpc: dial tcp 104.36.82.16:4306: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 16:01:09.607211200 (190h35m4.038291332s) [seg_pos={0 0}, upload=198245]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 16 16:01:09.843745933 (190h35m4.274826064s) [upload=198245]: Done uploading segments... Aug 16 16:01:09.843772213 (190h35m4.274852344s) [upload=198245]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 6.151225565s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)