Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.536092913s FAIL (2025-08-25 03:40:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Aug 25 03:17:25.962233411 (393h51m20.393313542s) [upload=409681]: Starting upload Aug 25 03:17:25.962276153 (393h51m20.393356284s) [upload=409681]: Done starting upload Aug 25 03:17:25.962388208 (393h51m20.393468359s) [upload=409681]: Uploading segments... Aug 25 03:17:25.962446609 (393h51m20.39352674s) [upload=409681]: Got next segment. Inline: false Aug 25 03:17:25.962477408 (393h51m20.393557539s) [upload=409681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:17:25.962481546 (393h51m20.393561667s) [upload=409681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:17:26.414369804 (393h51m20.845449925s) [seg_pos={0 0}, upload=409681]: Begin upload segment... Aug 25 03:17:26.414416634 (393h51m20.845496765s) [seg_pos={0 0}, upload=409681]: Done begin upload segment. Aug 25 03:17:26.414446431 (393h51m20.845526562s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414455198 (393h51m20.845535329s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414487420 (393h51m20.845567541s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414816980 (393h51m20.845897111s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415094702 (393h51m20.846174873s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415211314 (393h51m20.846291445s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415744726 (393h51m20.846824857s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.416912930 (393h51m20.847993061s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.417127441 (393h51m20.848207572s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419245273 (393h51m20.850325414s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419666530 (393h51m20.850746661s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419932508 (393h51m20.851012639s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420190171 (393h51m20.851270302s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420510544 (393h51m20.851590675s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420735856 (393h51m20.851815987s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420989370 (393h51m20.852069502s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421212046 (393h51m20.852292167s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421414774 (393h51m20.852494905s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:27.588072299 (393h51m22.01915243s) [upload=409681]: Next returned nil segment Aug 25 03:17:27.588092738 (393h51m22.019172869s) [upload=409681]: Waiting for error group managing segments... Aug 25 03:17:27.917909945 (393h51m22.348990076s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.321409426 (393h51m22.752489607s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.352135587 (393h51m22.783215758s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.364441663 (393h51m22.795521794s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.405467832 (393h51m22.836547963s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.410774175 (393h51m22.841854306s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.415913338 (393h51m22.846993469s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.455424158 (393h51m22.886504289s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.489790754 (393h51m22.920870885s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.578955615 (393h51m24.010035766s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.880978180 (393h51m24.312058311s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:18:25.306818461 (393h52m19.737898592s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:121wUjWPiPFgX5PLRcNrs535zEL81fbPpquD9rhSaqkpWxjGZLY, address:45.91.116.10:4114): protocol: rpc: dial tcp 45.91.116.10:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306864459 (393h52m19.73794458s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12XLPBKV4R1CLgVK32nSQffKHxjhU7XsGo8MEsHaAbDGiusTTAv, address:77.90.31.16:4743): protocol: write tcp 188.166.225.170:59222->77.90.31.16:4743: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306904285 (393h52m19.737984416s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ, address:104.36.82.16:4328): protocol: rpc: dial tcp 104.36.82.16:4328: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306934373 (393h52m19.738014494s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: rpc: dial tcp 104.36.82.5:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306956545 (393h52m19.738036666s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12r5RXjrbpRcyxvcYhpoqQdgv8MD8zLmpninS7D3cdNCid4Rc8j, address:104.36.82.38:4719): protocol: rpc: dial tcp 104.36.82.38:4719: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306963759 (393h52m19.73804389s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:125NsQhq7kRmTjKS8nuc51PZddApFEi5C38FzC5pEX2QrWYKzSW, address:104.36.82.6:4880): protocol: rpc: dial tcp 104.36.82.6:4880: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306989077 (393h52m19.738069208s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:104.36.82.21:4477): protocol: write tcp 188.166.225.170:48118->104.36.82.21:4477: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.307007322 (393h52m19.738087454s) [seg_pos={0 0}, upload=409681]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 25 03:18:25.534777114 (393h52m19.965857235s) [upload=409681]: Done uploading segments... Aug 25 03:18:25.534801571 (393h52m19.965881702s) [upload=409681]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.536110648s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [......X......X...............X..................X...........] duration: 1.971041336s FAIL (2025-08-25 03:40:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 25 03:32:01.143205020 (394h5m55.574285151s) [upload=409921]: Starting upload Aug 25 03:32:01.143283010 (394h5m55.574363151s) [upload=409921]: Done starting upload Aug 25 03:32:01.143365568 (394h5m55.574445699s) [upload=409921]: Uploading segments... Aug 25 03:32:01.143425061 (394h5m55.574505202s) [upload=409921]: Got next segment. Inline: true Aug 25 03:32:01.143431243 (394h5m55.574511384s) [upload=409921]: Waiting for error group managing segments... Aug 25 03:32:01.143481219 (394h5m55.57456137s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:32:01.143487320 (394h5m55.574567481s) [upload=409921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 25 03:32:01.143491559 (394h5m55.5745717s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 25 03:32:01.447212806 (394h5m55.878292947s) [upload=409921]: Done uploading segments... Aug 25 03:32:01.447238395 (394h5m55.878318526s) [upload=409921]: Upload finished. err: <nil> Duration:* 1.971061073s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 476.337495ms target: https://eu1.storj.io/login FAIL (2025-08-25 03:40:34 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 476.364366ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.039474333s FAIL (2025-08-25 03:40:26 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.039492318s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................XXX.........................] duration: 5.736315688s FAIL (2025-08-25 03:40:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 25 03:18:58.489563607 (393h52m52.920643728s) [upload=409700]: Starting upload Aug 25 03:18:58.489622922 (393h52m52.920703043s) [upload=409700]: Done starting upload Aug 25 03:18:58.489679550 (393h52m52.920759681s) [upload=409700]: Uploading segments... Aug 25 03:18:58.489954276 (393h52m52.921034407s) [upload=409700]: Got next segment. Inline: false Aug 25 03:18:58.489980907 (393h52m52.921061038s) [upload=409700]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:18:58.489985295 (393h52m52.921065426s) [upload=409700]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:18:58.733585695 (393h52m53.164665826s) [seg_pos={0 0}, upload=409700]: Begin upload segment... Aug 25 03:18:58.733626764 (393h52m53.164706885s) [seg_pos={0 0}, upload=409700]: Done begin upload segment. Aug 25 03:18:58.733639999 (393h52m53.16472013s) [noise=true , address=104.36.82.16:4295 , piece_id=5QRXMDIVBAOPBR6MDLQQLFZ3QPKENUX3UUQLBZ7BLZLKERM7LYIQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.733647243 (393h52m53.164727374s) [noise=true , address=45.91.116.13:4155 , piece_id=2G765KQXTVXTCUUN6MULVA4OSCKDJ7XM25CY6NP4KRC6GZ5V6WYA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.733868557 (393h52m53.164948688s) [noise=true , address=45.91.116.5:4299 , piece_id=WM65QYKBUDVFGJU6DHKVBF75BHAHTNXHLNTT4IJMDMLPCVKUTDLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734002592 (393h52m53.165082724s) [noise=true , address=104.36.82.14:4221 , piece_id=J65LKYLRWV75ZSIO4RP4V46M6OU6GDC3NXD72SKFEFWC3LNFIYGQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734264414 (393h52m53.165344545s) [noise=true , address=104.36.82.9:4005 , piece_id=JOAHLNP7DPPHB3QE2W5MIVLKAOLXS3WDT3NJH4C6FBOMZP7HX6BQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734513861 (393h52m53.165593992s) [noise=true , address=45.91.116.8:4045 , piece_id=JFFYQZ4DVKJVYCHE6LDGOXZOLZBQSP42WNYMQH7QKQLJHQVMGAJA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734779650 (393h52m53.165859781s) [noise=true , address=77.90.31.11:4554 , piece_id=USHDI5HFQQEKIEX6T7MGHVC27EWZ3OA7GCIPG2YFRTAB7BGQZ47A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735033776 (393h52m53.166113927s) [noise=true , address=45.91.116.12:4225 , piece_id=DSLEPIVXUAAAWL2BDPM3CH4V6Y6DVAQ4JLI6GYR7TCT4DBTKUQ5A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735302881 (393h52m53.166383012s) [noise=true , address=104.36.82.12:4151 , piece_id=LA3SOGB5MZBGD27US5CWGOI7IFKP74FSGG5ZAHCFSICLL44TWOYQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735508504 (393h52m53.166588625s) [noise=true , address=45.91.116.4:4940 , piece_id=CDGC3XQGGBYVQEWDCTCW3AMUZHA2AJPFG4MUGWBIZTKB2G4C7YOA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735712805 (393h52m53.166792926s) [noise=true , address=77.90.31.21:4878 , piece_id=IESVPC4WUD74EQA36PDX3PDPXQMLQJOXHBJNFTL4KAYGLW23GOFA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735924701 (393h52m53.167004832s) [noise=true , address=77.90.31.19:4842 , piece_id=3WVVR5B6NF7TTIXNYOKYBICOPJRVFE3VHSEREAXLQW2XQM4OU47A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736104805 (393h52m53.167184926s) [noise=true , address=77.90.31.14:4663 , piece_id=3CH32MMOCUBFE2KMCYSB4332AQY4TJ2PQ7IDMAMQI7EI4YHK6ZLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736289549 (393h52m53.16736968s) [noise=true , address=104.36.82.9:4006 , piece_id=SNJIO7LYCXVXBD3AREJYKLSMLVTOUHQGFM3MREYUJFBK4YB3MJLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736597137 (393h52m53.167677268s) [noise=true , address=45.91.116.6:4974 , piece_id=GMZ7KDUNI63OKFNLPVRQFHKZAD5FRONV6EAGUHQWFC2CEAV4CN6A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736833951 (393h52m53.167914082s) [noise=true , address=104.36.82.6:4876 , piece_id=6IH7K7LNDBX23XDK2TE4FRGOLNF3WZBYZVOZIFNXKZCAPREOMYYA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:19:00.476181002 (393h52m54.907261133s) [upload=409700]: Next returned nil segment Aug 25 03:19:00.476200320 (393h52m54.907280441s) [upload=409700]: Waiting for error group managing segments... Aug 25 03:19:00.755872911 (393h52m55.186953042s) [noise=true , address=45.91.116.5:4299 , piece_id=WM65QYKBUDVFGJU6DHKVBF75BHAHTNXHLNTT4IJMDMLPCVKUTDLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.766788056 (393h52m55.197868187s) [noise=true , address=77.90.31.21:4878 , piece_id=IESVPC4WUD74EQA36PDX3PDPXQMLQJOXHBJNFTL4KAYGLW23GOFA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.776992220 (393h52m55.208072351s) [noise=true , address=45.91.116.8:4045 , piece_id=JFFYQZ4DVKJVYCHE6LDGOXZOLZBQSP42WNYMQH7QKQLJHQVMGAJA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.805184684 (393h52m55.236264815s) [noise=true , address=104.36.82.9:4005 , piece_id=JOAHLNP7DPPHB3QE2W5MIVLKAOLXS3WDT3NJH4C6FBOMZP7HX6BQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.831528319 (393h52m55.26260845s) [noise=true , address=104.36.82.16:4295 , piece_id=5QRXMDIVBAOPBR6MDLQQLFZ3QPKENUX3UUQLBZ7BLZLKERM7LYIQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.836356277 (393h52m55.267436408s) [noise=true , address=45.91.116.4:4940 , piece_id=CDGC3XQGGBYVQEWDCTCW3AMUZHA2AJPFG4MUGWBIZTKB2G4C7YOA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.839769588 (393h52m55.270849719s) [noise=true , address=77.90.31.14:4663 , piece_id=3CH32MMOCUBFE2KMCYSB4332AQY4TJ2PQ7IDMAMQI7EI4YHK6ZLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.863991474 (393h52m55.295071605s) [noise=true , address=104.36.82.6:4876 , piece_id=6IH7K7LNDBX23XDK2TE4FRGOLNF3WZBYZVOZIFNXKZCAPREOMYYA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.916115397 (393h52m55.347195518s) [noise=true , address=77.90.31.11:4554 , piece_id=USHDI5HFQQEKIEX6T7MGHVC27EWZ3OA7GCIPG2YFRTAB7BGQZ47A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:01.008754882 (393h52m55.439835013s) [noise=true , address=77.90.31.19:4842 , piece_id=3WVVR5B6NF7TTIXNYOKYBICOPJRVFE3VHSEREAXLQW2XQM4OU47A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:27.839565824 (393h53m22.270645955s) [noise=true , address=104.36.82.12:4151 , piece_id=LA3SOGB5MZBGD27US5CWGOI7IFKP74FSGG5ZAHCFSICLL44TWOYQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839642881 (393h53m22.270723013s) [noise=true , address=45.91.116.13:4155 , piece_id=2G765KQXTVXTCUUN6MULVA4OSCKDJ7XM25CY6NP4KRC6GZ5V6WYA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: rpc: dial tcp 45.91.116.13:4155: i/o timeout; conn closed: op Aug 25 03:19:27.839686084 (393h53m22.270766215s) [noise=true , address=104.36.82.9:4006 , piece_id=SNJIO7LYCXVXBD3AREJYKLSMLVTOUHQGFM3MREYUJFBK4YB3MJLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:58626->104.36.82.9:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839729918 (393h53m22.270810049s) [noise=true , address=45.91.116.12:4225 , piece_id=DSLEPIVXUAAAWL2BDPM3CH4V6Y6DVAQ4JLI6GYR7TCT4DBTKUQ5A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: context deadline exceeded; context deadline exceeded Aug 25 03:19:27.839748123 (393h53m22.270828254s) [noise=true , address=45.91.116.6:4974 , piece_id=GMZ7KDUNI63OKFNLPVRQFHKZAD5FRONV6EAGUHQWFC2CEAV4CN6A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839830360 (393h53m22.270910491s) [noise=true , address=104.36.82.14:4221 , piece_id=J65LKYLRWV75ZSIO4RP4V46M6OU6GDC3NXD72SKFEFWC3LNFIYGQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: context deadline exceeded; context deadline exceeded Aug 25 03:19:27.839846671 (393h53m22.270926802s) [seg_pos={0 0}, upload=409700]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 25 03:19:28.068351319 (393h53m22.499431451s) [upload=409700]: Done uploading segments... Aug 25 03:19:28.068375766 (393h53m22.499455897s) [upload=409700]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.73634276s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..............................X............................] duration: 5.827382765s FAIL (2025-08-25 03:40:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Aug 25 03:17:25.962233411 (393h51m20.393313542s) [upload=409681]: Starting upload Aug 25 03:17:25.962276153 (393h51m20.393356284s) [upload=409681]: Done starting upload Aug 25 03:17:25.962388208 (393h51m20.393468359s) [upload=409681]: Uploading segments... Aug 25 03:17:25.962446609 (393h51m20.39352674s) [upload=409681]: Got next segment. Inline: false Aug 25 03:17:25.962477408 (393h51m20.393557539s) [upload=409681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:17:25.962481546 (393h51m20.393561667s) [upload=409681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:17:26.414369804 (393h51m20.845449925s) [seg_pos={0 0}, upload=409681]: Begin upload segment... Aug 25 03:17:26.414416634 (393h51m20.845496765s) [seg_pos={0 0}, upload=409681]: Done begin upload segment. Aug 25 03:17:26.414446431 (393h51m20.845526562s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414455198 (393h51m20.845535329s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414487420 (393h51m20.845567541s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414816980 (393h51m20.845897111s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415094702 (393h51m20.846174873s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415211314 (393h51m20.846291445s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415744726 (393h51m20.846824857s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.416912930 (393h51m20.847993061s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.417127441 (393h51m20.848207572s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419245273 (393h51m20.850325414s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419666530 (393h51m20.850746661s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419932508 (393h51m20.851012639s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420190171 (393h51m20.851270302s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420510544 (393h51m20.851590675s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420735856 (393h51m20.851815987s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420989370 (393h51m20.852069502s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421212046 (393h51m20.852292167s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421414774 (393h51m20.852494905s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:27.588072299 (393h51m22.01915243s) [upload=409681]: Next returned nil segment Aug 25 03:17:27.588092738 (393h51m22.019172869s) [upload=409681]: Waiting for error group managing segments... Aug 25 03:17:27.917909945 (393h51m22.348990076s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.321409426 (393h51m22.752489607s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.352135587 (393h51m22.783215758s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.364441663 (393h51m22.795521794s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.405467832 (393h51m22.836547963s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.410774175 (393h51m22.841854306s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.415913338 (393h51m22.846993469s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.455424158 (393h51m22.886504289s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.489790754 (393h51m22.920870885s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.578955615 (393h51m24.010035766s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.880978180 (393h51m24.312058311s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:18:25.306818461 (393h52m19.737898592s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:121wUjWPiPFgX5PLRcNrs535zEL81fbPpquD9rhSaqkpWxjGZLY, address:45.91.116.10:4114): protocol: rpc: dial tcp 45.91.116.10:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306864459 (393h52m19.73794458s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12XLPBKV4R1CLgVK32nSQffKHxjhU7XsGo8MEsHaAbDGiusTTAv, address:77.90.31.16:4743): protocol: write tcp 188.166.225.170:59222->77.90.31.16:4743: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306904285 (393h52m19.737984416s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ, address:104.36.82.16:4328): protocol: rpc: dial tcp 104.36.82.16:4328: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306934373 (393h52m19.738014494s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: rpc: dial tcp 104.36.82.5:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306956545 (393h52m19.738036666s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12r5RXjrbpRcyxvcYhpoqQdgv8MD8zLmpninS7D3cdNCid4Rc8j, address:104.36.82.38:4719): protocol: rpc: dial tcp 104.36.82.38:4719: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306963759 (393h52m19.73804389s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:125NsQhq7kRmTjKS8nuc51PZddApFEi5C38FzC5pEX2QrWYKzSW, address:104.36.82.6:4880): protocol: rpc: dial tcp 104.36.82.6:4880: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306989077 (393h52m19.738069208s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:104.36.82.21:4477): protocol: write tcp 188.166.225.170:48118->104.36.82.21:4477: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.307007322 (393h52m19.738087454s) [seg_pos={0 0}, upload=409681]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 25 03:18:25.534777114 (393h52m19.965857235s) [upload=409681]: Done uploading segments... Aug 25 03:18:25.534801571 (393h52m19.965881702s) [upload=409681]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.827410007s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.449745298s FAIL (2025-08-25 03:40:19 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.44977227s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 4.72478486s FAIL (2025-08-25 03:39:50 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.724813163s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.......X......X...............X..................X..........] duration: 1.874566361s FAIL (2025-08-25 03:39:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 25 03:32:01.143205020 (394h5m55.574285151s) [upload=409921]: Starting upload Aug 25 03:32:01.143283010 (394h5m55.574363151s) [upload=409921]: Done starting upload Aug 25 03:32:01.143365568 (394h5m55.574445699s) [upload=409921]: Uploading segments... Aug 25 03:32:01.143425061 (394h5m55.574505202s) [upload=409921]: Got next segment. Inline: true Aug 25 03:32:01.143431243 (394h5m55.574511384s) [upload=409921]: Waiting for error group managing segments... Aug 25 03:32:01.143481219 (394h5m55.57456137s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:32:01.143487320 (394h5m55.574567481s) [upload=409921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 25 03:32:01.143491559 (394h5m55.5745717s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 25 03:32:01.447212806 (394h5m55.878292947s) [upload=409921]: Done uploading segments... Aug 25 03:32:01.447238395 (394h5m55.878318526s) [upload=409921]: Upload finished. err: <nil> Duration:* 1.874599905s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 3.882793739s FAIL (2025-08-25 03:39:44 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 3.882818957s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 500.732083ms target: https://eu1.storj.io/login FAIL (2025-08-25 03:39:40 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 500.758615ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.065814433s FAIL (2025-08-25 03:39:33 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.065841405s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.490114109s FAIL (2025-08-25 03:39:22 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.490138336s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 464.73679ms target: https://eu1.storj.io/login FAIL (2025-08-25 03:39:13 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 464.793037ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................XXX........................] duration: 5.601925359s FAIL (2025-08-25 03:39:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 25 03:18:58.489563607 (393h52m52.920643728s) [upload=409700]: Starting upload Aug 25 03:18:58.489622922 (393h52m52.920703043s) [upload=409700]: Done starting upload Aug 25 03:18:58.489679550 (393h52m52.920759681s) [upload=409700]: Uploading segments... Aug 25 03:18:58.489954276 (393h52m52.921034407s) [upload=409700]: Got next segment. Inline: false Aug 25 03:18:58.489980907 (393h52m52.921061038s) [upload=409700]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:18:58.489985295 (393h52m52.921065426s) [upload=409700]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:18:58.733585695 (393h52m53.164665826s) [seg_pos={0 0}, upload=409700]: Begin upload segment... Aug 25 03:18:58.733626764 (393h52m53.164706885s) [seg_pos={0 0}, upload=409700]: Done begin upload segment. Aug 25 03:18:58.733639999 (393h52m53.16472013s) [noise=true , address=104.36.82.16:4295 , piece_id=5QRXMDIVBAOPBR6MDLQQLFZ3QPKENUX3UUQLBZ7BLZLKERM7LYIQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.733647243 (393h52m53.164727374s) [noise=true , address=45.91.116.13:4155 , piece_id=2G765KQXTVXTCUUN6MULVA4OSCKDJ7XM25CY6NP4KRC6GZ5V6WYA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.733868557 (393h52m53.164948688s) [noise=true , address=45.91.116.5:4299 , piece_id=WM65QYKBUDVFGJU6DHKVBF75BHAHTNXHLNTT4IJMDMLPCVKUTDLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734002592 (393h52m53.165082724s) [noise=true , address=104.36.82.14:4221 , piece_id=J65LKYLRWV75ZSIO4RP4V46M6OU6GDC3NXD72SKFEFWC3LNFIYGQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734264414 (393h52m53.165344545s) [noise=true , address=104.36.82.9:4005 , piece_id=JOAHLNP7DPPHB3QE2W5MIVLKAOLXS3WDT3NJH4C6FBOMZP7HX6BQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734513861 (393h52m53.165593992s) [noise=true , address=45.91.116.8:4045 , piece_id=JFFYQZ4DVKJVYCHE6LDGOXZOLZBQSP42WNYMQH7QKQLJHQVMGAJA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734779650 (393h52m53.165859781s) [noise=true , address=77.90.31.11:4554 , piece_id=USHDI5HFQQEKIEX6T7MGHVC27EWZ3OA7GCIPG2YFRTAB7BGQZ47A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735033776 (393h52m53.166113927s) [noise=true , address=45.91.116.12:4225 , piece_id=DSLEPIVXUAAAWL2BDPM3CH4V6Y6DVAQ4JLI6GYR7TCT4DBTKUQ5A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735302881 (393h52m53.166383012s) [noise=true , address=104.36.82.12:4151 , piece_id=LA3SOGB5MZBGD27US5CWGOI7IFKP74FSGG5ZAHCFSICLL44TWOYQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735508504 (393h52m53.166588625s) [noise=true , address=45.91.116.4:4940 , piece_id=CDGC3XQGGBYVQEWDCTCW3AMUZHA2AJPFG4MUGWBIZTKB2G4C7YOA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735712805 (393h52m53.166792926s) [noise=true , address=77.90.31.21:4878 , piece_id=IESVPC4WUD74EQA36PDX3PDPXQMLQJOXHBJNFTL4KAYGLW23GOFA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735924701 (393h52m53.167004832s) [noise=true , address=77.90.31.19:4842 , piece_id=3WVVR5B6NF7TTIXNYOKYBICOPJRVFE3VHSEREAXLQW2XQM4OU47A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736104805 (393h52m53.167184926s) [noise=true , address=77.90.31.14:4663 , piece_id=3CH32MMOCUBFE2KMCYSB4332AQY4TJ2PQ7IDMAMQI7EI4YHK6ZLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736289549 (393h52m53.16736968s) [noise=true , address=104.36.82.9:4006 , piece_id=SNJIO7LYCXVXBD3AREJYKLSMLVTOUHQGFM3MREYUJFBK4YB3MJLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736597137 (393h52m53.167677268s) [noise=true , address=45.91.116.6:4974 , piece_id=GMZ7KDUNI63OKFNLPVRQFHKZAD5FRONV6EAGUHQWFC2CEAV4CN6A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736833951 (393h52m53.167914082s) [noise=true , address=104.36.82.6:4876 , piece_id=6IH7K7LNDBX23XDK2TE4FRGOLNF3WZBYZVOZIFNXKZCAPREOMYYA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:19:00.476181002 (393h52m54.907261133s) [upload=409700]: Next returned nil segment Aug 25 03:19:00.476200320 (393h52m54.907280441s) [upload=409700]: Waiting for error group managing segments... Aug 25 03:19:00.755872911 (393h52m55.186953042s) [noise=true , address=45.91.116.5:4299 , piece_id=WM65QYKBUDVFGJU6DHKVBF75BHAHTNXHLNTT4IJMDMLPCVKUTDLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.766788056 (393h52m55.197868187s) [noise=true , address=77.90.31.21:4878 , piece_id=IESVPC4WUD74EQA36PDX3PDPXQMLQJOXHBJNFTL4KAYGLW23GOFA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.776992220 (393h52m55.208072351s) [noise=true , address=45.91.116.8:4045 , piece_id=JFFYQZ4DVKJVYCHE6LDGOXZOLZBQSP42WNYMQH7QKQLJHQVMGAJA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.805184684 (393h52m55.236264815s) [noise=true , address=104.36.82.9:4005 , piece_id=JOAHLNP7DPPHB3QE2W5MIVLKAOLXS3WDT3NJH4C6FBOMZP7HX6BQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.831528319 (393h52m55.26260845s) [noise=true , address=104.36.82.16:4295 , piece_id=5QRXMDIVBAOPBR6MDLQQLFZ3QPKENUX3UUQLBZ7BLZLKERM7LYIQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.836356277 (393h52m55.267436408s) [noise=true , address=45.91.116.4:4940 , piece_id=CDGC3XQGGBYVQEWDCTCW3AMUZHA2AJPFG4MUGWBIZTKB2G4C7YOA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.839769588 (393h52m55.270849719s) [noise=true , address=77.90.31.14:4663 , piece_id=3CH32MMOCUBFE2KMCYSB4332AQY4TJ2PQ7IDMAMQI7EI4YHK6ZLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.863991474 (393h52m55.295071605s) [noise=true , address=104.36.82.6:4876 , piece_id=6IH7K7LNDBX23XDK2TE4FRGOLNF3WZBYZVOZIFNXKZCAPREOMYYA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.916115397 (393h52m55.347195518s) [noise=true , address=77.90.31.11:4554 , piece_id=USHDI5HFQQEKIEX6T7MGHVC27EWZ3OA7GCIPG2YFRTAB7BGQZ47A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:01.008754882 (393h52m55.439835013s) [noise=true , address=77.90.31.19:4842 , piece_id=3WVVR5B6NF7TTIXNYOKYBICOPJRVFE3VHSEREAXLQW2XQM4OU47A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:27.839565824 (393h53m22.270645955s) [noise=true , address=104.36.82.12:4151 , piece_id=LA3SOGB5MZBGD27US5CWGOI7IFKP74FSGG5ZAHCFSICLL44TWOYQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839642881 (393h53m22.270723013s) [noise=true , address=45.91.116.13:4155 , piece_id=2G765KQXTVXTCUUN6MULVA4OSCKDJ7XM25CY6NP4KRC6GZ5V6WYA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: rpc: dial tcp 45.91.116.13:4155: i/o timeout; conn closed: op Aug 25 03:19:27.839686084 (393h53m22.270766215s) [noise=true , address=104.36.82.9:4006 , piece_id=SNJIO7LYCXVXBD3AREJYKLSMLVTOUHQGFM3MREYUJFBK4YB3MJLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:58626->104.36.82.9:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839729918 (393h53m22.270810049s) [noise=true , address=45.91.116.12:4225 , piece_id=DSLEPIVXUAAAWL2BDPM3CH4V6Y6DVAQ4JLI6GYR7TCT4DBTKUQ5A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: context deadline exceeded; context deadline exceeded Aug 25 03:19:27.839748123 (393h53m22.270828254s) [noise=true , address=45.91.116.6:4974 , piece_id=GMZ7KDUNI63OKFNLPVRQFHKZAD5FRONV6EAGUHQWFC2CEAV4CN6A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839830360 (393h53m22.270910491s) [noise=true , address=104.36.82.14:4221 , piece_id=J65LKYLRWV75ZSIO4RP4V46M6OU6GDC3NXD72SKFEFWC3LNFIYGQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: context deadline exceeded; context deadline exceeded Aug 25 03:19:27.839846671 (393h53m22.270926802s) [seg_pos={0 0}, upload=409700]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 25 03:19:28.068351319 (393h53m22.499431451s) [upload=409700]: Done uploading segments... Aug 25 03:19:28.068375766 (393h53m22.499455897s) [upload=409700]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.601957379s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.39499062s FAIL (2025-08-25 03:39:07 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.395015247s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..............................X...........................] duration: 5.70532932s FAIL (2025-08-25 03:39:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Aug 25 03:17:25.962233411 (393h51m20.393313542s) [upload=409681]: Starting upload Aug 25 03:17:25.962276153 (393h51m20.393356284s) [upload=409681]: Done starting upload Aug 25 03:17:25.962388208 (393h51m20.393468359s) [upload=409681]: Uploading segments... Aug 25 03:17:25.962446609 (393h51m20.39352674s) [upload=409681]: Got next segment. Inline: false Aug 25 03:17:25.962477408 (393h51m20.393557539s) [upload=409681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:17:25.962481546 (393h51m20.393561667s) [upload=409681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:17:26.414369804 (393h51m20.845449925s) [seg_pos={0 0}, upload=409681]: Begin upload segment... Aug 25 03:17:26.414416634 (393h51m20.845496765s) [seg_pos={0 0}, upload=409681]: Done begin upload segment. Aug 25 03:17:26.414446431 (393h51m20.845526562s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414455198 (393h51m20.845535329s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414487420 (393h51m20.845567541s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414816980 (393h51m20.845897111s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415094702 (393h51m20.846174873s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415211314 (393h51m20.846291445s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415744726 (393h51m20.846824857s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.416912930 (393h51m20.847993061s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.417127441 (393h51m20.848207572s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419245273 (393h51m20.850325414s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419666530 (393h51m20.850746661s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419932508 (393h51m20.851012639s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420190171 (393h51m20.851270302s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420510544 (393h51m20.851590675s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420735856 (393h51m20.851815987s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420989370 (393h51m20.852069502s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421212046 (393h51m20.852292167s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421414774 (393h51m20.852494905s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:27.588072299 (393h51m22.01915243s) [upload=409681]: Next returned nil segment Aug 25 03:17:27.588092738 (393h51m22.019172869s) [upload=409681]: Waiting for error group managing segments... Aug 25 03:17:27.917909945 (393h51m22.348990076s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.321409426 (393h51m22.752489607s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.352135587 (393h51m22.783215758s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.364441663 (393h51m22.795521794s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.405467832 (393h51m22.836547963s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.410774175 (393h51m22.841854306s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.415913338 (393h51m22.846993469s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.455424158 (393h51m22.886504289s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.489790754 (393h51m22.920870885s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.578955615 (393h51m24.010035766s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.880978180 (393h51m24.312058311s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:18:25.306818461 (393h52m19.737898592s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:121wUjWPiPFgX5PLRcNrs535zEL81fbPpquD9rhSaqkpWxjGZLY, address:45.91.116.10:4114): protocol: rpc: dial tcp 45.91.116.10:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306864459 (393h52m19.73794458s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12XLPBKV4R1CLgVK32nSQffKHxjhU7XsGo8MEsHaAbDGiusTTAv, address:77.90.31.16:4743): protocol: write tcp 188.166.225.170:59222->77.90.31.16:4743: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306904285 (393h52m19.737984416s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ, address:104.36.82.16:4328): protocol: rpc: dial tcp 104.36.82.16:4328: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306934373 (393h52m19.738014494s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: rpc: dial tcp 104.36.82.5:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306956545 (393h52m19.738036666s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12r5RXjrbpRcyxvcYhpoqQdgv8MD8zLmpninS7D3cdNCid4Rc8j, address:104.36.82.38:4719): protocol: rpc: dial tcp 104.36.82.38:4719: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306963759 (393h52m19.73804389s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:125NsQhq7kRmTjKS8nuc51PZddApFEi5C38FzC5pEX2QrWYKzSW, address:104.36.82.6:4880): protocol: rpc: dial tcp 104.36.82.6:4880: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306989077 (393h52m19.738069208s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:104.36.82.21:4477): protocol: write tcp 188.166.225.170:48118->104.36.82.21:4477: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.307007322 (393h52m19.738087454s) [seg_pos={0 0}, upload=409681]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 25 03:18:25.534777114 (393h52m19.965857235s) [upload=409681]: Done uploading segments... Aug 25 03:18:25.534801571 (393h52m19.965881702s) [upload=409681]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.705354649s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.046499065s FAIL (2025-08-25 03:38:46 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.046524354s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 483.173254ms target: https://eu1.storj.io/login FAIL (2025-08-25 03:38:45 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 483.204623ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 3.932111144s FAIL (2025-08-25 03:38:43 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 3.932137304s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [........X......X...............X..................X.........] duration: 1.816242237s FAIL (2025-08-25 03:38:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 25 03:32:01.143205020 (394h5m55.574285151s) [upload=409921]: Starting upload Aug 25 03:32:01.143283010 (394h5m55.574363151s) [upload=409921]: Done starting upload Aug 25 03:32:01.143365568 (394h5m55.574445699s) [upload=409921]: Uploading segments... Aug 25 03:32:01.143425061 (394h5m55.574505202s) [upload=409921]: Got next segment. Inline: true Aug 25 03:32:01.143431243 (394h5m55.574511384s) [upload=409921]: Waiting for error group managing segments... Aug 25 03:32:01.143481219 (394h5m55.57456137s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:32:01.143487320 (394h5m55.574567481s) [upload=409921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 25 03:32:01.143491559 (394h5m55.5745717s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 25 03:32:01.447212806 (394h5m55.878292947s) [upload=409921]: Done uploading segments... Aug 25 03:32:01.447238395 (394h5m55.878318526s) [upload=409921]: Upload finished. err: <nil> Duration:* 1.816267906s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.374598783s FAIL (2025-08-25 03:38:39 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.374620385s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.120280957s FAIL (2025-08-25 03:38:30 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.120308019s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..............................X..........................] duration: 5.591213451s FAIL (2025-08-25 03:38:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Aug 25 03:17:25.962233411 (393h51m20.393313542s) [upload=409681]: Starting upload Aug 25 03:17:25.962276153 (393h51m20.393356284s) [upload=409681]: Done starting upload Aug 25 03:17:25.962388208 (393h51m20.393468359s) [upload=409681]: Uploading segments... Aug 25 03:17:25.962446609 (393h51m20.39352674s) [upload=409681]: Got next segment. Inline: false Aug 25 03:17:25.962477408 (393h51m20.393557539s) [upload=409681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:17:25.962481546 (393h51m20.393561667s) [upload=409681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:17:26.414369804 (393h51m20.845449925s) [seg_pos={0 0}, upload=409681]: Begin upload segment... Aug 25 03:17:26.414416634 (393h51m20.845496765s) [seg_pos={0 0}, upload=409681]: Done begin upload segment. Aug 25 03:17:26.414446431 (393h51m20.845526562s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414455198 (393h51m20.845535329s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414487420 (393h51m20.845567541s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414816980 (393h51m20.845897111s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415094702 (393h51m20.846174873s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415211314 (393h51m20.846291445s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415744726 (393h51m20.846824857s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.416912930 (393h51m20.847993061s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.417127441 (393h51m20.848207572s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419245273 (393h51m20.850325414s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419666530 (393h51m20.850746661s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419932508 (393h51m20.851012639s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420190171 (393h51m20.851270302s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420510544 (393h51m20.851590675s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420735856 (393h51m20.851815987s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420989370 (393h51m20.852069502s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421212046 (393h51m20.852292167s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421414774 (393h51m20.852494905s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:27.588072299 (393h51m22.01915243s) [upload=409681]: Next returned nil segment Aug 25 03:17:27.588092738 (393h51m22.019172869s) [upload=409681]: Waiting for error group managing segments... Aug 25 03:17:27.917909945 (393h51m22.348990076s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.321409426 (393h51m22.752489607s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.352135587 (393h51m22.783215758s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.364441663 (393h51m22.795521794s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.405467832 (393h51m22.836547963s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.410774175 (393h51m22.841854306s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.415913338 (393h51m22.846993469s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.455424158 (393h51m22.886504289s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.489790754 (393h51m22.920870885s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.578955615 (393h51m24.010035766s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.880978180 (393h51m24.312058311s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:18:25.306818461 (393h52m19.737898592s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:121wUjWPiPFgX5PLRcNrs535zEL81fbPpquD9rhSaqkpWxjGZLY, address:45.91.116.10:4114): protocol: rpc: dial tcp 45.91.116.10:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306864459 (393h52m19.73794458s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12XLPBKV4R1CLgVK32nSQffKHxjhU7XsGo8MEsHaAbDGiusTTAv, address:77.90.31.16:4743): protocol: write tcp 188.166.225.170:59222->77.90.31.16:4743: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306904285 (393h52m19.737984416s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ, address:104.36.82.16:4328): protocol: rpc: dial tcp 104.36.82.16:4328: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306934373 (393h52m19.738014494s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: rpc: dial tcp 104.36.82.5:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306956545 (393h52m19.738036666s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12r5RXjrbpRcyxvcYhpoqQdgv8MD8zLmpninS7D3cdNCid4Rc8j, address:104.36.82.38:4719): protocol: rpc: dial tcp 104.36.82.38:4719: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306963759 (393h52m19.73804389s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:125NsQhq7kRmTjKS8nuc51PZddApFEi5C38FzC5pEX2QrWYKzSW, address:104.36.82.6:4880): protocol: rpc: dial tcp 104.36.82.6:4880: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306989077 (393h52m19.738069208s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:104.36.82.21:4477): protocol: write tcp 188.166.225.170:48118->104.36.82.21:4477: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.307007322 (393h52m19.738087454s) [seg_pos={0 0}, upload=409681]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 25 03:18:25.534777114 (393h52m19.965857235s) [upload=409681]: Done uploading segments... Aug 25 03:18:25.534801571 (393h52m19.965881702s) [upload=409681]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.591242106s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.384940651s FAIL (2025-08-25 03:38:06 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.384969054s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 4.901504063s FAIL (2025-08-25 03:38:04 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.901529511s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 477.552201ms target: https://eu1.storj.io/login FAIL (2025-08-25 03:38:00 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 477.611895ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................XXX.......................] duration: 5.648529117s FAIL (2025-08-25 03:37:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 25 03:18:58.489563607 (393h52m52.920643728s) [upload=409700]: Starting upload Aug 25 03:18:58.489622922 (393h52m52.920703043s) [upload=409700]: Done starting upload Aug 25 03:18:58.489679550 (393h52m52.920759681s) [upload=409700]: Uploading segments... Aug 25 03:18:58.489954276 (393h52m52.921034407s) [upload=409700]: Got next segment. Inline: false Aug 25 03:18:58.489980907 (393h52m52.921061038s) [upload=409700]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:18:58.489985295 (393h52m52.921065426s) [upload=409700]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:18:58.733585695 (393h52m53.164665826s) [seg_pos={0 0}, upload=409700]: Begin upload segment... Aug 25 03:18:58.733626764 (393h52m53.164706885s) [seg_pos={0 0}, upload=409700]: Done begin upload segment. Aug 25 03:18:58.733639999 (393h52m53.16472013s) [noise=true , address=104.36.82.16:4295 , piece_id=5QRXMDIVBAOPBR6MDLQQLFZ3QPKENUX3UUQLBZ7BLZLKERM7LYIQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.733647243 (393h52m53.164727374s) [noise=true , address=45.91.116.13:4155 , piece_id=2G765KQXTVXTCUUN6MULVA4OSCKDJ7XM25CY6NP4KRC6GZ5V6WYA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.733868557 (393h52m53.164948688s) [noise=true , address=45.91.116.5:4299 , piece_id=WM65QYKBUDVFGJU6DHKVBF75BHAHTNXHLNTT4IJMDMLPCVKUTDLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734002592 (393h52m53.165082724s) [noise=true , address=104.36.82.14:4221 , piece_id=J65LKYLRWV75ZSIO4RP4V46M6OU6GDC3NXD72SKFEFWC3LNFIYGQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734264414 (393h52m53.165344545s) [noise=true , address=104.36.82.9:4005 , piece_id=JOAHLNP7DPPHB3QE2W5MIVLKAOLXS3WDT3NJH4C6FBOMZP7HX6BQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734513861 (393h52m53.165593992s) [noise=true , address=45.91.116.8:4045 , piece_id=JFFYQZ4DVKJVYCHE6LDGOXZOLZBQSP42WNYMQH7QKQLJHQVMGAJA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734779650 (393h52m53.165859781s) [noise=true , address=77.90.31.11:4554 , piece_id=USHDI5HFQQEKIEX6T7MGHVC27EWZ3OA7GCIPG2YFRTAB7BGQZ47A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735033776 (393h52m53.166113927s) [noise=true , address=45.91.116.12:4225 , piece_id=DSLEPIVXUAAAWL2BDPM3CH4V6Y6DVAQ4JLI6GYR7TCT4DBTKUQ5A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735302881 (393h52m53.166383012s) [noise=true , address=104.36.82.12:4151 , piece_id=LA3SOGB5MZBGD27US5CWGOI7IFKP74FSGG5ZAHCFSICLL44TWOYQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735508504 (393h52m53.166588625s) [noise=true , address=45.91.116.4:4940 , piece_id=CDGC3XQGGBYVQEWDCTCW3AMUZHA2AJPFG4MUGWBIZTKB2G4C7YOA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735712805 (393h52m53.166792926s) [noise=true , address=77.90.31.21:4878 , piece_id=IESVPC4WUD74EQA36PDX3PDPXQMLQJOXHBJNFTL4KAYGLW23GOFA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735924701 (393h52m53.167004832s) [noise=true , address=77.90.31.19:4842 , piece_id=3WVVR5B6NF7TTIXNYOKYBICOPJRVFE3VHSEREAXLQW2XQM4OU47A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736104805 (393h52m53.167184926s) [noise=true , address=77.90.31.14:4663 , piece_id=3CH32MMOCUBFE2KMCYSB4332AQY4TJ2PQ7IDMAMQI7EI4YHK6ZLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736289549 (393h52m53.16736968s) [noise=true , address=104.36.82.9:4006 , piece_id=SNJIO7LYCXVXBD3AREJYKLSMLVTOUHQGFM3MREYUJFBK4YB3MJLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736597137 (393h52m53.167677268s) [noise=true , address=45.91.116.6:4974 , piece_id=GMZ7KDUNI63OKFNLPVRQFHKZAD5FRONV6EAGUHQWFC2CEAV4CN6A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736833951 (393h52m53.167914082s) [noise=true , address=104.36.82.6:4876 , piece_id=6IH7K7LNDBX23XDK2TE4FRGOLNF3WZBYZVOZIFNXKZCAPREOMYYA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:19:00.476181002 (393h52m54.907261133s) [upload=409700]: Next returned nil segment Aug 25 03:19:00.476200320 (393h52m54.907280441s) [upload=409700]: Waiting for error group managing segments... Aug 25 03:19:00.755872911 (393h52m55.186953042s) [noise=true , address=45.91.116.5:4299 , piece_id=WM65QYKBUDVFGJU6DHKVBF75BHAHTNXHLNTT4IJMDMLPCVKUTDLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.766788056 (393h52m55.197868187s) [noise=true , address=77.90.31.21:4878 , piece_id=IESVPC4WUD74EQA36PDX3PDPXQMLQJOXHBJNFTL4KAYGLW23GOFA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.776992220 (393h52m55.208072351s) [noise=true , address=45.91.116.8:4045 , piece_id=JFFYQZ4DVKJVYCHE6LDGOXZOLZBQSP42WNYMQH7QKQLJHQVMGAJA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.805184684 (393h52m55.236264815s) [noise=true , address=104.36.82.9:4005 , piece_id=JOAHLNP7DPPHB3QE2W5MIVLKAOLXS3WDT3NJH4C6FBOMZP7HX6BQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.831528319 (393h52m55.26260845s) [noise=true , address=104.36.82.16:4295 , piece_id=5QRXMDIVBAOPBR6MDLQQLFZ3QPKENUX3UUQLBZ7BLZLKERM7LYIQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.836356277 (393h52m55.267436408s) [noise=true , address=45.91.116.4:4940 , piece_id=CDGC3XQGGBYVQEWDCTCW3AMUZHA2AJPFG4MUGWBIZTKB2G4C7YOA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.839769588 (393h52m55.270849719s) [noise=true , address=77.90.31.14:4663 , piece_id=3CH32MMOCUBFE2KMCYSB4332AQY4TJ2PQ7IDMAMQI7EI4YHK6ZLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.863991474 (393h52m55.295071605s) [noise=true , address=104.36.82.6:4876 , piece_id=6IH7K7LNDBX23XDK2TE4FRGOLNF3WZBYZVOZIFNXKZCAPREOMYYA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.916115397 (393h52m55.347195518s) [noise=true , address=77.90.31.11:4554 , piece_id=USHDI5HFQQEKIEX6T7MGHVC27EWZ3OA7GCIPG2YFRTAB7BGQZ47A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:01.008754882 (393h52m55.439835013s) [noise=true , address=77.90.31.19:4842 , piece_id=3WVVR5B6NF7TTIXNYOKYBICOPJRVFE3VHSEREAXLQW2XQM4OU47A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:27.839565824 (393h53m22.270645955s) [noise=true , address=104.36.82.12:4151 , piece_id=LA3SOGB5MZBGD27US5CWGOI7IFKP74FSGG5ZAHCFSICLL44TWOYQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839642881 (393h53m22.270723013s) [noise=true , address=45.91.116.13:4155 , piece_id=2G765KQXTVXTCUUN6MULVA4OSCKDJ7XM25CY6NP4KRC6GZ5V6WYA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: rpc: dial tcp 45.91.116.13:4155: i/o timeout; conn closed: op Aug 25 03:19:27.839686084 (393h53m22.270766215s) [noise=true , address=104.36.82.9:4006 , piece_id=SNJIO7LYCXVXBD3AREJYKLSMLVTOUHQGFM3MREYUJFBK4YB3MJLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:58626->104.36.82.9:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839729918 (393h53m22.270810049s) [noise=true , address=45.91.116.12:4225 , piece_id=DSLEPIVXUAAAWL2BDPM3CH4V6Y6DVAQ4JLI6GYR7TCT4DBTKUQ5A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: context deadline exceeded; context deadline exceeded Aug 25 03:19:27.839748123 (393h53m22.270828254s) [noise=true , address=45.91.116.6:4974 , piece_id=GMZ7KDUNI63OKFNLPVRQFHKZAD5FRONV6EAGUHQWFC2CEAV4CN6A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839830360 (393h53m22.270910491s) [noise=true , address=104.36.82.14:4221 , piece_id=J65LKYLRWV75ZSIO4RP4V46M6OU6GDC3NXD72SKFEFWC3LNFIYGQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: context deadline exceeded; context deadline exceeded Aug 25 03:19:27.839846671 (393h53m22.270926802s) [seg_pos={0 0}, upload=409700]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 25 03:19:28.068351319 (393h53m22.499431451s) [upload=409700]: Done uploading segments... Aug 25 03:19:28.068375766 (393h53m22.499455897s) [upload=409700]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.648557412s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.........X......X...............X..................X........] duration: 1.957985335s FAIL (2025-08-25 03:37:51 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 25 03:32:01.143205020 (394h5m55.574285151s) [upload=409921]: Starting upload Aug 25 03:32:01.143283010 (394h5m55.574363151s) [upload=409921]: Done starting upload Aug 25 03:32:01.143365568 (394h5m55.574445699s) [upload=409921]: Uploading segments... Aug 25 03:32:01.143425061 (394h5m55.574505202s) [upload=409921]: Got next segment. Inline: true Aug 25 03:32:01.143431243 (394h5m55.574511384s) [upload=409921]: Waiting for error group managing segments... Aug 25 03:32:01.143481219 (394h5m55.57456137s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:32:01.143487320 (394h5m55.574567481s) [upload=409921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 25 03:32:01.143491559 (394h5m55.5745717s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 25 03:32:01.447212806 (394h5m55.878292947s) [upload=409921]: Done uploading segments... Aug 25 03:32:01.447238395 (394h5m55.878318526s) [upload=409921]: Upload finished. err: <nil> Duration:* 1.958015671s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.414301543s FAIL (2025-08-25 03:37:45 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.414329888s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 3.919052368s FAIL (2025-08-25 03:37:29 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 3.919080101s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X..............................X.........................] duration: 5.466425672s FAIL (2025-08-25 03:37:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Aug 25 03:17:25.962233411 (393h51m20.393313542s) [upload=409681]: Starting upload Aug 25 03:17:25.962276153 (393h51m20.393356284s) [upload=409681]: Done starting upload Aug 25 03:17:25.962388208 (393h51m20.393468359s) [upload=409681]: Uploading segments... Aug 25 03:17:25.962446609 (393h51m20.39352674s) [upload=409681]: Got next segment. Inline: false Aug 25 03:17:25.962477408 (393h51m20.393557539s) [upload=409681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:17:25.962481546 (393h51m20.393561667s) [upload=409681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:17:26.414369804 (393h51m20.845449925s) [seg_pos={0 0}, upload=409681]: Begin upload segment... Aug 25 03:17:26.414416634 (393h51m20.845496765s) [seg_pos={0 0}, upload=409681]: Done begin upload segment. Aug 25 03:17:26.414446431 (393h51m20.845526562s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414455198 (393h51m20.845535329s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414487420 (393h51m20.845567541s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414816980 (393h51m20.845897111s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415094702 (393h51m20.846174873s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415211314 (393h51m20.846291445s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415744726 (393h51m20.846824857s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.416912930 (393h51m20.847993061s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.417127441 (393h51m20.848207572s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419245273 (393h51m20.850325414s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419666530 (393h51m20.850746661s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419932508 (393h51m20.851012639s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420190171 (393h51m20.851270302s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420510544 (393h51m20.851590675s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420735856 (393h51m20.851815987s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420989370 (393h51m20.852069502s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421212046 (393h51m20.852292167s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421414774 (393h51m20.852494905s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:27.588072299 (393h51m22.01915243s) [upload=409681]: Next returned nil segment Aug 25 03:17:27.588092738 (393h51m22.019172869s) [upload=409681]: Waiting for error group managing segments... Aug 25 03:17:27.917909945 (393h51m22.348990076s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.321409426 (393h51m22.752489607s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.352135587 (393h51m22.783215758s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.364441663 (393h51m22.795521794s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.405467832 (393h51m22.836547963s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.410774175 (393h51m22.841854306s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.415913338 (393h51m22.846993469s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.455424158 (393h51m22.886504289s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.489790754 (393h51m22.920870885s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.578955615 (393h51m24.010035766s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.880978180 (393h51m24.312058311s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:18:25.306818461 (393h52m19.737898592s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:121wUjWPiPFgX5PLRcNrs535zEL81fbPpquD9rhSaqkpWxjGZLY, address:45.91.116.10:4114): protocol: rpc: dial tcp 45.91.116.10:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306864459 (393h52m19.73794458s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12XLPBKV4R1CLgVK32nSQffKHxjhU7XsGo8MEsHaAbDGiusTTAv, address:77.90.31.16:4743): protocol: write tcp 188.166.225.170:59222->77.90.31.16:4743: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306904285 (393h52m19.737984416s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ, address:104.36.82.16:4328): protocol: rpc: dial tcp 104.36.82.16:4328: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306934373 (393h52m19.738014494s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: rpc: dial tcp 104.36.82.5:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306956545 (393h52m19.738036666s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12r5RXjrbpRcyxvcYhpoqQdgv8MD8zLmpninS7D3cdNCid4Rc8j, address:104.36.82.38:4719): protocol: rpc: dial tcp 104.36.82.38:4719: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306963759 (393h52m19.73804389s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:125NsQhq7kRmTjKS8nuc51PZddApFEi5C38FzC5pEX2QrWYKzSW, address:104.36.82.6:4880): protocol: rpc: dial tcp 104.36.82.6:4880: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306989077 (393h52m19.738069208s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:104.36.82.21:4477): protocol: write tcp 188.166.225.170:48118->104.36.82.21:4477: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.307007322 (393h52m19.738087454s) [seg_pos={0 0}, upload=409681]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 25 03:18:25.534777114 (393h52m19.965857235s) [upload=409681]: Done uploading segments... Aug 25 03:18:25.534801571 (393h52m19.965881702s) [upload=409681]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.466455268s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........X......X...............X..................X.......] duration: 1.852934364s FAIL (2025-08-25 03:37:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 25 03:32:01.143205020 (394h5m55.574285151s) [upload=409921]: Starting upload Aug 25 03:32:01.143283010 (394h5m55.574363151s) [upload=409921]: Done starting upload Aug 25 03:32:01.143365568 (394h5m55.574445699s) [upload=409921]: Uploading segments... Aug 25 03:32:01.143425061 (394h5m55.574505202s) [upload=409921]: Got next segment. Inline: true Aug 25 03:32:01.143431243 (394h5m55.574511384s) [upload=409921]: Waiting for error group managing segments... Aug 25 03:32:01.143481219 (394h5m55.57456137s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:32:01.143487320 (394h5m55.574567481s) [upload=409921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 25 03:32:01.143491559 (394h5m55.5745717s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 25 03:32:01.447212806 (394h5m55.878292947s) [upload=409921]: Done uploading segments... Aug 25 03:32:01.447238395 (394h5m55.878318526s) [upload=409921]: Upload finished. err: <nil> Duration:* 1.852966356s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X..............................X........................] duration: 5.785935809s FAIL (2025-08-25 03:37:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Aug 25 03:17:25.962233411 (393h51m20.393313542s) [upload=409681]: Starting upload Aug 25 03:17:25.962276153 (393h51m20.393356284s) [upload=409681]: Done starting upload Aug 25 03:17:25.962388208 (393h51m20.393468359s) [upload=409681]: Uploading segments... Aug 25 03:17:25.962446609 (393h51m20.39352674s) [upload=409681]: Got next segment. Inline: false Aug 25 03:17:25.962477408 (393h51m20.393557539s) [upload=409681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:17:25.962481546 (393h51m20.393561667s) [upload=409681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:17:26.414369804 (393h51m20.845449925s) [seg_pos={0 0}, upload=409681]: Begin upload segment... Aug 25 03:17:26.414416634 (393h51m20.845496765s) [seg_pos={0 0}, upload=409681]: Done begin upload segment. Aug 25 03:17:26.414446431 (393h51m20.845526562s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414455198 (393h51m20.845535329s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414487420 (393h51m20.845567541s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414816980 (393h51m20.845897111s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415094702 (393h51m20.846174873s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415211314 (393h51m20.846291445s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415744726 (393h51m20.846824857s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.416912930 (393h51m20.847993061s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.417127441 (393h51m20.848207572s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419245273 (393h51m20.850325414s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419666530 (393h51m20.850746661s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419932508 (393h51m20.851012639s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420190171 (393h51m20.851270302s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420510544 (393h51m20.851590675s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420735856 (393h51m20.851815987s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420989370 (393h51m20.852069502s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421212046 (393h51m20.852292167s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421414774 (393h51m20.852494905s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:27.588072299 (393h51m22.01915243s) [upload=409681]: Next returned nil segment Aug 25 03:17:27.588092738 (393h51m22.019172869s) [upload=409681]: Waiting for error group managing segments... Aug 25 03:17:27.917909945 (393h51m22.348990076s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.321409426 (393h51m22.752489607s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.352135587 (393h51m22.783215758s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.364441663 (393h51m22.795521794s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.405467832 (393h51m22.836547963s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.410774175 (393h51m22.841854306s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.415913338 (393h51m22.846993469s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.455424158 (393h51m22.886504289s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.489790754 (393h51m22.920870885s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.578955615 (393h51m24.010035766s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.880978180 (393h51m24.312058311s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:18:25.306818461 (393h52m19.737898592s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:121wUjWPiPFgX5PLRcNrs535zEL81fbPpquD9rhSaqkpWxjGZLY, address:45.91.116.10:4114): protocol: rpc: dial tcp 45.91.116.10:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306864459 (393h52m19.73794458s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12XLPBKV4R1CLgVK32nSQffKHxjhU7XsGo8MEsHaAbDGiusTTAv, address:77.90.31.16:4743): protocol: write tcp 188.166.225.170:59222->77.90.31.16:4743: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306904285 (393h52m19.737984416s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ, address:104.36.82.16:4328): protocol: rpc: dial tcp 104.36.82.16:4328: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306934373 (393h52m19.738014494s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: rpc: dial tcp 104.36.82.5:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306956545 (393h52m19.738036666s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12r5RXjrbpRcyxvcYhpoqQdgv8MD8zLmpninS7D3cdNCid4Rc8j, address:104.36.82.38:4719): protocol: rpc: dial tcp 104.36.82.38:4719: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306963759 (393h52m19.73804389s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:125NsQhq7kRmTjKS8nuc51PZddApFEi5C38FzC5pEX2QrWYKzSW, address:104.36.82.6:4880): protocol: rpc: dial tcp 104.36.82.6:4880: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306989077 (393h52m19.738069208s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:104.36.82.21:4477): protocol: write tcp 188.166.225.170:48118->104.36.82.21:4477: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.307007322 (393h52m19.738087454s) [seg_pos={0 0}, upload=409681]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 25 03:18:25.534777114 (393h52m19.965857235s) [upload=409681]: Done uploading segments... Aug 25 03:18:25.534801571 (393h52m19.965881702s) [upload=409681]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.785969784s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 467.934721ms target: https://eu1.storj.io/login FAIL (2025-08-25 03:37:03 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 467.964618ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 4.9981951s FAIL (2025-08-25 03:37:00 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.998232972s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.429625662s FAIL (2025-08-25 03:36:52 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.429651731s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................XXX......................] duration: 5.701629531s FAIL (2025-08-25 03:36:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 25 03:18:58.489563607 (393h52m52.920643728s) [upload=409700]: Starting upload Aug 25 03:18:58.489622922 (393h52m52.920703043s) [upload=409700]: Done starting upload Aug 25 03:18:58.489679550 (393h52m52.920759681s) [upload=409700]: Uploading segments... Aug 25 03:18:58.489954276 (393h52m52.921034407s) [upload=409700]: Got next segment. Inline: false Aug 25 03:18:58.489980907 (393h52m52.921061038s) [upload=409700]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:18:58.489985295 (393h52m52.921065426s) [upload=409700]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:18:58.733585695 (393h52m53.164665826s) [seg_pos={0 0}, upload=409700]: Begin upload segment... Aug 25 03:18:58.733626764 (393h52m53.164706885s) [seg_pos={0 0}, upload=409700]: Done begin upload segment. Aug 25 03:18:58.733639999 (393h52m53.16472013s) [noise=true , address=104.36.82.16:4295 , piece_id=5QRXMDIVBAOPBR6MDLQQLFZ3QPKENUX3UUQLBZ7BLZLKERM7LYIQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.733647243 (393h52m53.164727374s) [noise=true , address=45.91.116.13:4155 , piece_id=2G765KQXTVXTCUUN6MULVA4OSCKDJ7XM25CY6NP4KRC6GZ5V6WYA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.733868557 (393h52m53.164948688s) [noise=true , address=45.91.116.5:4299 , piece_id=WM65QYKBUDVFGJU6DHKVBF75BHAHTNXHLNTT4IJMDMLPCVKUTDLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734002592 (393h52m53.165082724s) [noise=true , address=104.36.82.14:4221 , piece_id=J65LKYLRWV75ZSIO4RP4V46M6OU6GDC3NXD72SKFEFWC3LNFIYGQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734264414 (393h52m53.165344545s) [noise=true , address=104.36.82.9:4005 , piece_id=JOAHLNP7DPPHB3QE2W5MIVLKAOLXS3WDT3NJH4C6FBOMZP7HX6BQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734513861 (393h52m53.165593992s) [noise=true , address=45.91.116.8:4045 , piece_id=JFFYQZ4DVKJVYCHE6LDGOXZOLZBQSP42WNYMQH7QKQLJHQVMGAJA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734779650 (393h52m53.165859781s) [noise=true , address=77.90.31.11:4554 , piece_id=USHDI5HFQQEKIEX6T7MGHVC27EWZ3OA7GCIPG2YFRTAB7BGQZ47A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735033776 (393h52m53.166113927s) [noise=true , address=45.91.116.12:4225 , piece_id=DSLEPIVXUAAAWL2BDPM3CH4V6Y6DVAQ4JLI6GYR7TCT4DBTKUQ5A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735302881 (393h52m53.166383012s) [noise=true , address=104.36.82.12:4151 , piece_id=LA3SOGB5MZBGD27US5CWGOI7IFKP74FSGG5ZAHCFSICLL44TWOYQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735508504 (393h52m53.166588625s) [noise=true , address=45.91.116.4:4940 , piece_id=CDGC3XQGGBYVQEWDCTCW3AMUZHA2AJPFG4MUGWBIZTKB2G4C7YOA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735712805 (393h52m53.166792926s) [noise=true , address=77.90.31.21:4878 , piece_id=IESVPC4WUD74EQA36PDX3PDPXQMLQJOXHBJNFTL4KAYGLW23GOFA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735924701 (393h52m53.167004832s) [noise=true , address=77.90.31.19:4842 , piece_id=3WVVR5B6NF7TTIXNYOKYBICOPJRVFE3VHSEREAXLQW2XQM4OU47A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736104805 (393h52m53.167184926s) [noise=true , address=77.90.31.14:4663 , piece_id=3CH32MMOCUBFE2KMCYSB4332AQY4TJ2PQ7IDMAMQI7EI4YHK6ZLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736289549 (393h52m53.16736968s) [noise=true , address=104.36.82.9:4006 , piece_id=SNJIO7LYCXVXBD3AREJYKLSMLVTOUHQGFM3MREYUJFBK4YB3MJLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736597137 (393h52m53.167677268s) [noise=true , address=45.91.116.6:4974 , piece_id=GMZ7KDUNI63OKFNLPVRQFHKZAD5FRONV6EAGUHQWFC2CEAV4CN6A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736833951 (393h52m53.167914082s) [noise=true , address=104.36.82.6:4876 , piece_id=6IH7K7LNDBX23XDK2TE4FRGOLNF3WZBYZVOZIFNXKZCAPREOMYYA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:19:00.476181002 (393h52m54.907261133s) [upload=409700]: Next returned nil segment Aug 25 03:19:00.476200320 (393h52m54.907280441s) [upload=409700]: Waiting for error group managing segments... Aug 25 03:19:00.755872911 (393h52m55.186953042s) [noise=true , address=45.91.116.5:4299 , piece_id=WM65QYKBUDVFGJU6DHKVBF75BHAHTNXHLNTT4IJMDMLPCVKUTDLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.766788056 (393h52m55.197868187s) [noise=true , address=77.90.31.21:4878 , piece_id=IESVPC4WUD74EQA36PDX3PDPXQMLQJOXHBJNFTL4KAYGLW23GOFA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.776992220 (393h52m55.208072351s) [noise=true , address=45.91.116.8:4045 , piece_id=JFFYQZ4DVKJVYCHE6LDGOXZOLZBQSP42WNYMQH7QKQLJHQVMGAJA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.805184684 (393h52m55.236264815s) [noise=true , address=104.36.82.9:4005 , piece_id=JOAHLNP7DPPHB3QE2W5MIVLKAOLXS3WDT3NJH4C6FBOMZP7HX6BQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.831528319 (393h52m55.26260845s) [noise=true , address=104.36.82.16:4295 , piece_id=5QRXMDIVBAOPBR6MDLQQLFZ3QPKENUX3UUQLBZ7BLZLKERM7LYIQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.836356277 (393h52m55.267436408s) [noise=true , address=45.91.116.4:4940 , piece_id=CDGC3XQGGBYVQEWDCTCW3AMUZHA2AJPFG4MUGWBIZTKB2G4C7YOA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.839769588 (393h52m55.270849719s) [noise=true , address=77.90.31.14:4663 , piece_id=3CH32MMOCUBFE2KMCYSB4332AQY4TJ2PQ7IDMAMQI7EI4YHK6ZLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.863991474 (393h52m55.295071605s) [noise=true , address=104.36.82.6:4876 , piece_id=6IH7K7LNDBX23XDK2TE4FRGOLNF3WZBYZVOZIFNXKZCAPREOMYYA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.916115397 (393h52m55.347195518s) [noise=true , address=77.90.31.11:4554 , piece_id=USHDI5HFQQEKIEX6T7MGHVC27EWZ3OA7GCIPG2YFRTAB7BGQZ47A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:01.008754882 (393h52m55.439835013s) [noise=true , address=77.90.31.19:4842 , piece_id=3WVVR5B6NF7TTIXNYOKYBICOPJRVFE3VHSEREAXLQW2XQM4OU47A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:27.839565824 (393h53m22.270645955s) [noise=true , address=104.36.82.12:4151 , piece_id=LA3SOGB5MZBGD27US5CWGOI7IFKP74FSGG5ZAHCFSICLL44TWOYQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839642881 (393h53m22.270723013s) [noise=true , address=45.91.116.13:4155 , piece_id=2G765KQXTVXTCUUN6MULVA4OSCKDJ7XM25CY6NP4KRC6GZ5V6WYA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: rpc: dial tcp 45.91.116.13:4155: i/o timeout; conn closed: op Aug 25 03:19:27.839686084 (393h53m22.270766215s) [noise=true , address=104.36.82.9:4006 , piece_id=SNJIO7LYCXVXBD3AREJYKLSMLVTOUHQGFM3MREYUJFBK4YB3MJLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:58626->104.36.82.9:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839729918 (393h53m22.270810049s) [noise=true , address=45.91.116.12:4225 , piece_id=DSLEPIVXUAAAWL2BDPM3CH4V6Y6DVAQ4JLI6GYR7TCT4DBTKUQ5A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: context deadline exceeded; context deadline exceeded Aug 25 03:19:27.839748123 (393h53m22.270828254s) [noise=true , address=45.91.116.6:4974 , piece_id=GMZ7KDUNI63OKFNLPVRQFHKZAD5FRONV6EAGUHQWFC2CEAV4CN6A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839830360 (393h53m22.270910491s) [noise=true , address=104.36.82.14:4221 , piece_id=J65LKYLRWV75ZSIO4RP4V46M6OU6GDC3NXD72SKFEFWC3LNFIYGQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: context deadline exceeded; context deadline exceeded Aug 25 03:19:27.839846671 (393h53m22.270926802s) [seg_pos={0 0}, upload=409700]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 25 03:19:28.068351319 (393h53m22.499431451s) [upload=409700]: Done uploading segments... Aug 25 03:19:28.068375766 (393h53m22.499455897s) [upload=409700]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.701655611s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...........X......X...............X..................X......] duration: 1.881873535s FAIL (2025-08-25 03:36:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 25 03:32:01.143205020 (394h5m55.574285151s) [upload=409921]: Starting upload Aug 25 03:32:01.143283010 (394h5m55.574363151s) [upload=409921]: Done starting upload Aug 25 03:32:01.143365568 (394h5m55.574445699s) [upload=409921]: Uploading segments... Aug 25 03:32:01.143425061 (394h5m55.574505202s) [upload=409921]: Got next segment. Inline: true Aug 25 03:32:01.143431243 (394h5m55.574511384s) [upload=409921]: Waiting for error group managing segments... Aug 25 03:32:01.143481219 (394h5m55.57456137s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:32:01.143487320 (394h5m55.574567481s) [upload=409921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 25 03:32:01.143491559 (394h5m55.5745717s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 25 03:32:01.447212806 (394h5m55.878292947s) [upload=409921]: Done uploading segments... Aug 25 03:32:01.447238395 (394h5m55.878318526s) [upload=409921]: Upload finished. err: <nil> Duration:* 1.881895667s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 4.097369123s FAIL (2025-08-25 03:36:36 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 4.097400692s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.413100271s FAIL (2025-08-25 03:36:28 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.413124349s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [............X......X...............X..................X.....] duration: 1.82982623s FAIL (2025-08-25 03:36:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 25 03:32:01.143205020 (394h5m55.574285151s) [upload=409921]: Starting upload Aug 25 03:32:01.143283010 (394h5m55.574363151s) [upload=409921]: Done starting upload Aug 25 03:32:01.143365568 (394h5m55.574445699s) [upload=409921]: Uploading segments... Aug 25 03:32:01.143425061 (394h5m55.574505202s) [upload=409921]: Got next segment. Inline: true Aug 25 03:32:01.143431243 (394h5m55.574511384s) [upload=409921]: Waiting for error group managing segments... Aug 25 03:32:01.143481219 (394h5m55.57456137s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:32:01.143487320 (394h5m55.574567481s) [upload=409921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 25 03:32:01.143491559 (394h5m55.5745717s) [upload=409921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 25 03:32:01.447212806 (394h5m55.878292947s) [upload=409921]: Done uploading segments... Aug 25 03:32:01.447238395 (394h5m55.878318526s) [upload=409921]: Upload finished. err: <nil> Duration:* 1.829850415s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X..............................X.......................] duration: 5.475179765s FAIL (2025-08-25 03:36:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Aug 25 03:17:25.962233411 (393h51m20.393313542s) [upload=409681]: Starting upload Aug 25 03:17:25.962276153 (393h51m20.393356284s) [upload=409681]: Done starting upload Aug 25 03:17:25.962388208 (393h51m20.393468359s) [upload=409681]: Uploading segments... Aug 25 03:17:25.962446609 (393h51m20.39352674s) [upload=409681]: Got next segment. Inline: false Aug 25 03:17:25.962477408 (393h51m20.393557539s) [upload=409681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:17:25.962481546 (393h51m20.393561667s) [upload=409681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:17:26.414369804 (393h51m20.845449925s) [seg_pos={0 0}, upload=409681]: Begin upload segment... Aug 25 03:17:26.414416634 (393h51m20.845496765s) [seg_pos={0 0}, upload=409681]: Done begin upload segment. Aug 25 03:17:26.414446431 (393h51m20.845526562s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414455198 (393h51m20.845535329s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414487420 (393h51m20.845567541s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414816980 (393h51m20.845897111s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415094702 (393h51m20.846174873s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415211314 (393h51m20.846291445s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415744726 (393h51m20.846824857s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.416912930 (393h51m20.847993061s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.417127441 (393h51m20.848207572s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419245273 (393h51m20.850325414s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419666530 (393h51m20.850746661s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419932508 (393h51m20.851012639s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420190171 (393h51m20.851270302s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420510544 (393h51m20.851590675s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420735856 (393h51m20.851815987s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420989370 (393h51m20.852069502s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421212046 (393h51m20.852292167s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421414774 (393h51m20.852494905s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:27.588072299 (393h51m22.01915243s) [upload=409681]: Next returned nil segment Aug 25 03:17:27.588092738 (393h51m22.019172869s) [upload=409681]: Waiting for error group managing segments... Aug 25 03:17:27.917909945 (393h51m22.348990076s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.321409426 (393h51m22.752489607s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.352135587 (393h51m22.783215758s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.364441663 (393h51m22.795521794s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.405467832 (393h51m22.836547963s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.410774175 (393h51m22.841854306s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.415913338 (393h51m22.846993469s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.455424158 (393h51m22.886504289s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.489790754 (393h51m22.920870885s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.578955615 (393h51m24.010035766s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.880978180 (393h51m24.312058311s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:18:25.306818461 (393h52m19.737898592s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:121wUjWPiPFgX5PLRcNrs535zEL81fbPpquD9rhSaqkpWxjGZLY, address:45.91.116.10:4114): protocol: rpc: dial tcp 45.91.116.10:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306864459 (393h52m19.73794458s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12XLPBKV4R1CLgVK32nSQffKHxjhU7XsGo8MEsHaAbDGiusTTAv, address:77.90.31.16:4743): protocol: write tcp 188.166.225.170:59222->77.90.31.16:4743: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306904285 (393h52m19.737984416s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ, address:104.36.82.16:4328): protocol: rpc: dial tcp 104.36.82.16:4328: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306934373 (393h52m19.738014494s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: rpc: dial tcp 104.36.82.5:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306956545 (393h52m19.738036666s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12r5RXjrbpRcyxvcYhpoqQdgv8MD8zLmpninS7D3cdNCid4Rc8j, address:104.36.82.38:4719): protocol: rpc: dial tcp 104.36.82.38:4719: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306963759 (393h52m19.73804389s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:125NsQhq7kRmTjKS8nuc51PZddApFEi5C38FzC5pEX2QrWYKzSW, address:104.36.82.6:4880): protocol: rpc: dial tcp 104.36.82.6:4880: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306989077 (393h52m19.738069208s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:104.36.82.21:4477): protocol: write tcp 188.166.225.170:48118->104.36.82.21:4477: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.307007322 (393h52m19.738087454s) [seg_pos={0 0}, upload=409681]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 25 03:18:25.534777114 (393h52m19.965857235s) [upload=409681]: Done uploading segments... Aug 25 03:18:25.534801571 (393h52m19.965881702s) [upload=409681]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.475202679s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.401262999s FAIL (2025-08-25 03:36:08 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.401291604s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 472.209359ms target: https://eu1.storj.io/login FAIL (2025-08-25 03:36:01 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 472.239426ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.22407638s FAIL (2025-08-25 03:35:56 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.224105636s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.65572679s FAIL (2025-08-25 03:35:50 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.655758602s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X..............................X......................] duration: 5.512447063s FAIL (2025-08-25 03:35:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Aug 25 03:17:25.962233411 (393h51m20.393313542s) [upload=409681]: Starting upload Aug 25 03:17:25.962276153 (393h51m20.393356284s) [upload=409681]: Done starting upload Aug 25 03:17:25.962388208 (393h51m20.393468359s) [upload=409681]: Uploading segments... Aug 25 03:17:25.962446609 (393h51m20.39352674s) [upload=409681]: Got next segment. Inline: false Aug 25 03:17:25.962477408 (393h51m20.393557539s) [upload=409681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:17:25.962481546 (393h51m20.393561667s) [upload=409681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:17:26.414369804 (393h51m20.845449925s) [seg_pos={0 0}, upload=409681]: Begin upload segment... Aug 25 03:17:26.414416634 (393h51m20.845496765s) [seg_pos={0 0}, upload=409681]: Done begin upload segment. Aug 25 03:17:26.414446431 (393h51m20.845526562s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414455198 (393h51m20.845535329s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414487420 (393h51m20.845567541s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.414816980 (393h51m20.845897111s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415094702 (393h51m20.846174873s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415211314 (393h51m20.846291445s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.415744726 (393h51m20.846824857s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.416912930 (393h51m20.847993061s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.417127441 (393h51m20.848207572s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419245273 (393h51m20.850325414s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419666530 (393h51m20.850746661s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.419932508 (393h51m20.851012639s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420190171 (393h51m20.851270302s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420510544 (393h51m20.851590675s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420735856 (393h51m20.851815987s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.420989370 (393h51m20.852069502s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421212046 (393h51m20.852292167s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:26.421414774 (393h51m20.852494905s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Uploading piece... Aug 25 03:17:27.588072299 (393h51m22.01915243s) [upload=409681]: Next returned nil segment Aug 25 03:17:27.588092738 (393h51m22.019172869s) [upload=409681]: Waiting for error group managing segments... Aug 25 03:17:27.917909945 (393h51m22.348990076s) [noise=true , address=79.127.205.245:4238 , piece_id=4WT4M3ALYXDQ7B26B2ADJYSR5AYBZ3OWDKOR3ULUCHTHQZW6HUKA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.321409426 (393h51m22.752489607s) [noise=true , address=104.36.82.13:4204 , piece_id=MRGWCLMF72KCUAOR5PCQLHZK7SNTMZXCCJEXQNXICPSWJV3NSEVQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.352135587 (393h51m22.783215758s) [noise=true , address=45.91.116.8:4070 , piece_id=2DSNEX2AMO2OMUTBXWM2D3KJWAWKVDQEG2YX6Q3CAGVNAPJVIB7A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.364441663 (393h51m22.795521794s) [noise=true , address=104.36.82.42:4816 , piece_id=DHZ77APUVCBSLKWKDYFTJG2ZL2DVYXKTIBQILNZZEBSUVBSX6VIA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.405467832 (393h51m22.836547963s) [noise=true , address=104.36.82.15:4270 , piece_id=3M3DMNJLGTERE4UPDWVLJ2WHSKECO4OG2DSTNQV2YASWZHTSZCOQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.410774175 (393h51m22.841854306s) [noise=true , address=79.127.201.215:4352 , piece_id=ODY6TKE6W6GK2SSVWBTYI5Q5A4OPFYHFEWPBPMQE5XXTZWKXP6HA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.415913338 (393h51m22.846993469s) [noise=true , address=104.36.82.45:4916 , piece_id=SQNXVUKS6PVTZGJE25YYAZ4LHQTPWPOVTADPJWUV7L6AYHAUEPCQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.455424158 (393h51m22.886504289s) [noise=true , address=104.36.82.60:4367 , piece_id=P3425AO3QOJXAPJQRBTP4UBKVFRIA7UVKKDPYCRTUVZSTGII6DDQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:28.489790754 (393h51m22.920870885s) [noise=true , address=104.36.82.14:4251 , piece_id=MFX2FBSYT3JN3L77J7WS4IWATQ7SOBDXRKZTBS6NW2P5Z7IQAIRQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.578955615 (393h51m24.010035766s) [noise=true , address=104.36.82.23:4455 , piece_id=MYAWGY2IN7QHNJGGSFERF34PNIVKBEUZDTV5Q2OEPC45NMQZVSAA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:17:29.880978180 (393h51m24.312058311s) [noise=true , address=45.91.116.7:4015 , piece_id=CPGYO5QQXYCYP2L5TVNVNBFJ4IOVLYVDXXUURPRYILYGRGWCTCQQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: <nil> Aug 25 03:18:25.306818461 (393h52m19.737898592s) [noise=true , address=45.91.116.10:4114 , piece_id=MW4ERIPVF3ABOW3ISK4CFEAMWTXMFSU7XCPCIKMU7LDP2RF67PKQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:121wUjWPiPFgX5PLRcNrs535zEL81fbPpquD9rhSaqkpWxjGZLY, address:45.91.116.10:4114): protocol: rpc: dial tcp 45.91.116.10:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306864459 (393h52m19.73794458s) [noise=true , address=77.90.31.16:4743 , piece_id=4HFG2MZ4BYIQVL7OZNU3RSLCW4G2E2JJA7HDQ5IPOWNUN5KBNBRA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12XLPBKV4R1CLgVK32nSQffKHxjhU7XsGo8MEsHaAbDGiusTTAv, address:77.90.31.16:4743): protocol: write tcp 188.166.225.170:59222->77.90.31.16:4743: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306904285 (393h52m19.737984416s) [noise=true , address=104.36.82.16:4328 , piece_id=WDJB7SZL6RBIZAPPW2QGU3TAYSQCOLGXLSIDQZ5KPE4CR2GNBFWA, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ, address:104.36.82.16:4328): protocol: rpc: dial tcp 104.36.82.16:4328: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306934373 (393h52m19.738014494s) [noise=true , address=104.36.82.5:4843 , piece_id=S4WSKOAFEFEH6TBNC3KDPHKWZF4RK4DAT72XBIWWSISNDHW6FTXQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: rpc: dial tcp 104.36.82.5:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306956545 (393h52m19.738036666s) [noise=true , address=104.36.82.38:4719 , piece_id=7N3IBAKDCVIMPJJY6G3ITLIT4LPPI6PWIIJ34SB75QBBL7D6JIHQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:12r5RXjrbpRcyxvcYhpoqQdgv8MD8zLmpninS7D3cdNCid4Rc8j, address:104.36.82.38:4719): protocol: rpc: dial tcp 104.36.82.38:4719: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306963759 (393h52m19.73804389s) [noise=true , address=104.36.82.6:4880 , piece_id=77QFZCG5AVEEJDT5TDQNP6MGJJTEEGLIYAPQUPSZG4VHM4TX4B2A, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:125NsQhq7kRmTjKS8nuc51PZddApFEi5C38FzC5pEX2QrWYKzSW, address:104.36.82.6:4880): protocol: rpc: dial tcp 104.36.82.6:4880: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.306989077 (393h52m19.738069208s) [noise=true , address=104.36.82.21:4477 , piece_id=4DQYWPNWCVXMIGMDILYSOZRGKTWDZDJ3WV3QRU453ULXKETXOARQ, seg_pos={0 0}, upload=409681]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:104.36.82.21:4477): protocol: write tcp 188.166.225.170:48118->104.36.82.21:4477: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:18:25.307007322 (393h52m19.738087454s) [seg_pos={0 0}, upload=409681]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 25 03:18:25.534777114 (393h52m19.965857235s) [upload=409681]: Done uploading segments... Aug 25 03:18:25.534801571 (393h52m19.965881702s) [upload=409681]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.512474696s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 4.918036045s FAIL (2025-08-25 03:35:41 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.918061323s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................XXX.....................] duration: 5.637096064s FAIL (2025-08-25 03:35:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 25 03:18:58.489563607 (393h52m52.920643728s) [upload=409700]: Starting upload Aug 25 03:18:58.489622922 (393h52m52.920703043s) [upload=409700]: Done starting upload Aug 25 03:18:58.489679550 (393h52m52.920759681s) [upload=409700]: Uploading segments... Aug 25 03:18:58.489954276 (393h52m52.921034407s) [upload=409700]: Got next segment. Inline: false Aug 25 03:18:58.489980907 (393h52m52.921061038s) [upload=409700]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 25 03:18:58.489985295 (393h52m52.921065426s) [upload=409700]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 25 03:18:58.733585695 (393h52m53.164665826s) [seg_pos={0 0}, upload=409700]: Begin upload segment... Aug 25 03:18:58.733626764 (393h52m53.164706885s) [seg_pos={0 0}, upload=409700]: Done begin upload segment. Aug 25 03:18:58.733639999 (393h52m53.16472013s) [noise=true , address=104.36.82.16:4295 , piece_id=5QRXMDIVBAOPBR6MDLQQLFZ3QPKENUX3UUQLBZ7BLZLKERM7LYIQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.733647243 (393h52m53.164727374s) [noise=true , address=45.91.116.13:4155 , piece_id=2G765KQXTVXTCUUN6MULVA4OSCKDJ7XM25CY6NP4KRC6GZ5V6WYA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.733868557 (393h52m53.164948688s) [noise=true , address=45.91.116.5:4299 , piece_id=WM65QYKBUDVFGJU6DHKVBF75BHAHTNXHLNTT4IJMDMLPCVKUTDLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734002592 (393h52m53.165082724s) [noise=true , address=104.36.82.14:4221 , piece_id=J65LKYLRWV75ZSIO4RP4V46M6OU6GDC3NXD72SKFEFWC3LNFIYGQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734264414 (393h52m53.165344545s) [noise=true , address=104.36.82.9:4005 , piece_id=JOAHLNP7DPPHB3QE2W5MIVLKAOLXS3WDT3NJH4C6FBOMZP7HX6BQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734513861 (393h52m53.165593992s) [noise=true , address=45.91.116.8:4045 , piece_id=JFFYQZ4DVKJVYCHE6LDGOXZOLZBQSP42WNYMQH7QKQLJHQVMGAJA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.734779650 (393h52m53.165859781s) [noise=true , address=77.90.31.11:4554 , piece_id=USHDI5HFQQEKIEX6T7MGHVC27EWZ3OA7GCIPG2YFRTAB7BGQZ47A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735033776 (393h52m53.166113927s) [noise=true , address=45.91.116.12:4225 , piece_id=DSLEPIVXUAAAWL2BDPM3CH4V6Y6DVAQ4JLI6GYR7TCT4DBTKUQ5A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735302881 (393h52m53.166383012s) [noise=true , address=104.36.82.12:4151 , piece_id=LA3SOGB5MZBGD27US5CWGOI7IFKP74FSGG5ZAHCFSICLL44TWOYQ, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735508504 (393h52m53.166588625s) [noise=true , address=45.91.116.4:4940 , piece_id=CDGC3XQGGBYVQEWDCTCW3AMUZHA2AJPFG4MUGWBIZTKB2G4C7YOA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735712805 (393h52m53.166792926s) [noise=true , address=77.90.31.21:4878 , piece_id=IESVPC4WUD74EQA36PDX3PDPXQMLQJOXHBJNFTL4KAYGLW23GOFA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.735924701 (393h52m53.167004832s) [noise=true , address=77.90.31.19:4842 , piece_id=3WVVR5B6NF7TTIXNYOKYBICOPJRVFE3VHSEREAXLQW2XQM4OU47A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736104805 (393h52m53.167184926s) [noise=true , address=77.90.31.14:4663 , piece_id=3CH32MMOCUBFE2KMCYSB4332AQY4TJ2PQ7IDMAMQI7EI4YHK6ZLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736289549 (393h52m53.16736968s) [noise=true , address=104.36.82.9:4006 , piece_id=SNJIO7LYCXVXBD3AREJYKLSMLVTOUHQGFM3MREYUJFBK4YB3MJLA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736597137 (393h52m53.167677268s) [noise=true , address=45.91.116.6:4974 , piece_id=GMZ7KDUNI63OKFNLPVRQFHKZAD5FRONV6EAGUHQWFC2CEAV4CN6A, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:18:58.736833951 (393h52m53.167914082s) [noise=true , address=104.36.82.6:4876 , piece_id=6IH7K7LNDBX23XDK2TE4FRGOLNF3WZBYZVOZIFNXKZCAPREOMYYA, seg_pos={0 0}, upload=409700]: Uploading piece... Aug 25 03:19:00.476181002 (393h52m54.907261133s) [upload=409700]: Next returned nil segment Aug 25 03:19:00.476200320 (393h52m54.907280441s) [upload=409700]: Waiting for error group managing segments... Aug 25 03:19:00.755872911 (393h52m55.186953042s) [noise=true , address=45.91.116.5:4299 , piece_id=WM65QYKBUDVFGJU6DHKVBF75BHAHTNXHLNTT4IJMDMLPCVKUTDLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.766788056 (393h52m55.197868187s) [noise=true , address=77.90.31.21:4878 , piece_id=IESVPC4WUD74EQA36PDX3PDPXQMLQJOXHBJNFTL4KAYGLW23GOFA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.776992220 (393h52m55.208072351s) [noise=true , address=45.91.116.8:4045 , piece_id=JFFYQZ4DVKJVYCHE6LDGOXZOLZBQSP42WNYMQH7QKQLJHQVMGAJA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.805184684 (393h52m55.236264815s) [noise=true , address=104.36.82.9:4005 , piece_id=JOAHLNP7DPPHB3QE2W5MIVLKAOLXS3WDT3NJH4C6FBOMZP7HX6BQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.831528319 (393h52m55.26260845s) [noise=true , address=104.36.82.16:4295 , piece_id=5QRXMDIVBAOPBR6MDLQQLFZ3QPKENUX3UUQLBZ7BLZLKERM7LYIQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.836356277 (393h52m55.267436408s) [noise=true , address=45.91.116.4:4940 , piece_id=CDGC3XQGGBYVQEWDCTCW3AMUZHA2AJPFG4MUGWBIZTKB2G4C7YOA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.839769588 (393h52m55.270849719s) [noise=true , address=77.90.31.14:4663 , piece_id=3CH32MMOCUBFE2KMCYSB4332AQY4TJ2PQ7IDMAMQI7EI4YHK6ZLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.863991474 (393h52m55.295071605s) [noise=true , address=104.36.82.6:4876 , piece_id=6IH7K7LNDBX23XDK2TE4FRGOLNF3WZBYZVOZIFNXKZCAPREOMYYA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:00.916115397 (393h52m55.347195518s) [noise=true , address=77.90.31.11:4554 , piece_id=USHDI5HFQQEKIEX6T7MGHVC27EWZ3OA7GCIPG2YFRTAB7BGQZ47A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:01.008754882 (393h52m55.439835013s) [noise=true , address=77.90.31.19:4842 , piece_id=3WVVR5B6NF7TTIXNYOKYBICOPJRVFE3VHSEREAXLQW2XQM4OU47A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: <nil> Aug 25 03:19:27.839565824 (393h53m22.270645955s) [noise=true , address=104.36.82.12:4151 , piece_id=LA3SOGB5MZBGD27US5CWGOI7IFKP74FSGG5ZAHCFSICLL44TWOYQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839642881 (393h53m22.270723013s) [noise=true , address=45.91.116.13:4155 , piece_id=2G765KQXTVXTCUUN6MULVA4OSCKDJ7XM25CY6NP4KRC6GZ5V6WYA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: rpc: dial tcp 45.91.116.13:4155: i/o timeout; conn closed: op Aug 25 03:19:27.839686084 (393h53m22.270766215s) [noise=true , address=104.36.82.9:4006 , piece_id=SNJIO7LYCXVXBD3AREJYKLSMLVTOUHQGFM3MREYUJFBK4YB3MJLA, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:58626->104.36.82.9:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839729918 (393h53m22.270810049s) [noise=true , address=45.91.116.12:4225 , piece_id=DSLEPIVXUAAAWL2BDPM3CH4V6Y6DVAQ4JLI6GYR7TCT4DBTKUQ5A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: context deadline exceeded; context deadline exceeded Aug 25 03:19:27.839748123 (393h53m22.270828254s) [noise=true , address=45.91.116.6:4974 , piece_id=GMZ7KDUNI63OKFNLPVRQFHKZAD5FRONV6EAGUHQWFC2CEAV4CN6A, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 25 03:19:27.839830360 (393h53m22.270910491s) [noise=true , address=104.36.82.14:4221 , piece_id=J65LKYLRWV75ZSIO4RP4V46M6OU6GDC3NXD72SKFEFWC3LNFIYGQ, seg_pos={0 0}, upload=409700]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: context deadline exceeded; context deadline exceeded Aug 25 03:19:27.839846671 (393h53m22.270926802s) [seg_pos={0 0}, upload=409700]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 25 03:19:28.068351319 (393h53m22.499431451s) [upload=409700]: Done uploading segments... Aug 25 03:19:28.068375766 (393h53m22.499455897s) [upload=409700]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.637122937s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)