Orbiter History

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 796.820711ms target: https://us1.storj.io/login FAIL (2025-09-16 16:12:37 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 796.850778ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [XXX.X..XX...................................................] duration: 9.565324991s FAIL (2025-09-16 16:12:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 9.565362784s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 17 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXX...........................................] duration: 6.280483279s FAIL (2025-09-16 16:12:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 36:23.221360752 (242h7m59.260933872s) [seg_pos={0 0}, upload=212758]: Begin upload segment... Sep 16 15:36:23.221427429 (242h7m59.261000559s) [seg_pos={0 0}, upload=212758]: Done begin upload segment. Sep 16 15:36:23.221471934 (242h7m59.261045064s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.221781523 (242h7m59.261354683s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222170084 (242h7m59.261743224s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222380514 (242h7m59.261953644s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222688551 (242h7m59.262261681s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222978833 (242h7m59.262551963s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223291669 (242h7m59.262864809s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223483104 (242h7m59.263056234s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223736246 (242h7m59.263309376s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223952347 (242h7m59.263525477s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224092043 (242h7m59.263665173s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224449704 (242h7m59.264022844s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224537902 (242h7m59.264111032s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224972190 (242h7m59.26454532s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225002056 (242h7m59.264575176s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225500526 (242h7m59.265073646s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225597541 (242h7m59.265170671s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225993644 (242h7m59.265566774s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:24.615369315 (242h8m0.654942445s) [upload=212758]: Next returned nil segment Sep 16 15:36:24.615405684 (242h8m0.654978814s) [upload=212758]: Waiting for error group managing segments... Sep 16 15:36:25.024537642 (242h8m1.064110772s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:07.783420676 (242h8m43.822993807s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:08.809548412 (242h8m44.849121542s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:09.417370950 (242h8m45.45694407s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:10.232853931 (242h8m46.272427062s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.251045936 (242h8m47.290619056s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.530631565 (242h8m47.570204695s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:12.400109127 (242h8m48.439682257s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:13.851060301 (242h8m49.890633431s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.293316228 (242h8m50.332889378s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.648711487 (242h8m50.688284607s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:15.183813908 (242h8m51.223387028s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:19.546944067 (242h8m55.586517197s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:21.938117511 (242h8m57.977690642s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938325488 (242h8m57.977898618s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12EsmL66Gn6K9cUxHqqGtuUZxUGyWCNL4oqyHzS1XUbWVnMMJbd, address:145.79.227.48:4314): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938466356 (242h8m57.978039476s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection Sep 16 15:37:21.938531310 (242h8m57.97810444s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12V9GZR1rr6VfEsK83Bonq2HyceXLcAKnYfoQmA1Qvt5h9u8sik, address:145.79.227.28:4742): protocol: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection Sep 16 15:37:21.938593297 (242h8m57.978166417s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection Sep 16 15:37:21.938613285 (242h8m57.978186405s) [seg_pos={0 0}, upload=212758]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 16 15:37:22.200860966 (242h8m58.240434086s) [upload=212758]: Done uploading segments... Sep 16 15:37:22.200890762 (242h8m58.240463893s) [upload=212758]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.280504178s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 748.542325ms target: https://us1.storj.io/login FAIL (2025-09-16 16:12:14 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 748.572592ms core satellite us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 12 out of 60 is failed (threshold=3) [..............................................XX.X.XXXXXXXXX] duration: 1m0.002109439s FAIL (2025-09-16 16:11:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== XXFWZUFU7L5LG57IZJASOGW3XSKWFB5335JANABQKQ, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.331512144 (242h42m26.371085274s) [noise=true , address=81.171.14.70:4029 , piece_id=Z6DXTIW4EJTX2PQWJSDETSEQEE3DP77OXNWW7YMUYEVMM5ZERB5A, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.331888531 (242h42m26.371461661s) [noise=true , address=89.149.201.147:4017 , piece_id=OPDS3GSZDWSF7HA3ZXU6AA4VXLFKFEDVN2AR62Q34TNXNOD5QSXQ, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.332283022 (242h42m26.371856152s) [noise=true , address=185.167.204.205:4030 , piece_id=TAGCXU46S677M3JO7BIQSH4KNYBQZSZVSZKMNO54ZAVLG5BOXBKQ, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.332672343 (242h42m26.372245473s) [noise=true , address=81.171.14.70:4037 , piece_id=FXJAHH5CCNHDCMGCFB4KFZVFSMCDGIZRG476ZGY233YTJKR7ODIA, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.333035926 (242h42m26.372609056s) [noise=true , address=89.149.201.147:4011 , piece_id=U5XKHDKNPCIZHO2SSCCPZYOFMGICFX65JIZ4HIPQQZB3OFV6SG5A, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.333471204 (242h42m26.373044334s) [noise=true , address=185.167.204.206:4038 , piece_id=QWVRZKVI3SDQIVXLAE62OSGVIXXZXQNPJTXG3AYGDZRZRXBJNCCA, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.333887517 (242h42m26.373460647s) [noise=true , address=185.167.204.205:4028 , piece_id=ZRNLEWOJYLDJJZ6X32ICTDHYCICG4XDWH2KW6Z2RAA5CBEHGVF5A, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.334247613 (242h42m26.373820743s) [noise=true , address=89.149.201.147:4007 , piece_id=XVDNKW23BOGM2ABQQUARBQJL76Z6JXFJOYMV7C2DFBABUYSX53YQ, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.334590236 (242h42m26.374163366s) [noise=true , address=185.167.204.206:4049 , piece_id=D6IGZ7FDPPCGYS6EFKFPOX6S5S64MR26TYNOIQT5AGSH55OFLMOQ, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.335425436 (242h42m26.374998556s) [noise=true , address=81.171.14.70:4028 , piece_id=K56FOSNYNRXVLN4ZALOXWTRSYUDQGEJSE2CXBIWVSO3N2BMIOGNQ, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.335831129 (242h42m26.375404259s) [noise=true , address=81.171.14.70:4030 , piece_id=AZ5TFWUXQDXGSI7TS67JS4WNLI3NDS6WGXBLE4RPG3VCTD7C66XQ, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:50.336334317 (242h42m26.375907447s) [noise=true , address=185.167.204.206:4056 , piece_id=5XZGML67X3A2ZXM2MX7SZF3WQ4JBOKTKJHUF4VXLJDATGPGVBEFA, seg_pos={0 0}, upload=213252]: Uploading piece... Sep 16 16:10:51.160054872 (242h42m27.199628002s) [upload=213252]: Next returned nil segment Sep 16 16:10:51.160109617 (242h42m27.199682747s) [upload=213252]: Waiting for error group managing segments... Sep 16 16:10:51.548126676 (242h42m27.587699796s) [noise=true , address=81.171.14.70:4037 , piece_id=FXJAHH5CCNHDCMGCFB4KFZVFSMCDGIZRG476ZGY233YTJKR7ODIA, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.552058422 (242h42m27.591631552s) [noise=true , address=81.171.14.70:4034 , piece_id=LHIKCX4LIP6E6RLYPO4WFAHL6OLL2EQ7FQTHXB5M4XV4DABXXWNQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.559937456 (242h42m27.599510586s) [noise=true , address=81.171.14.70:4026 , piece_id=BRK6ML3KNYGFBMHJR3AUDEG7MDEVTIRQITLNOWP42AW5Y5UEBAGQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.580884946 (242h42m27.620458076s) [noise=true , address=81.171.14.70:4029 , piece_id=Z6DXTIW4EJTX2PQWJSDETSEQEE3DP77OXNWW7YMUYEVMM5ZERB5A, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.581414835 (242h42m27.620987965s) [noise=true , address=89.149.201.147:4020 , piece_id=CCUAURJ2HBAIYMBMGYJ3WDZRDWUWETSGJWQYVA4NQMJMD2Q6OGIQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.587773133 (242h42m27.627346263s) [noise=true , address=89.149.201.147:4024 , piece_id=XGO7OZGAERSHEWWC3RPL24ZAEQNF2I6JKDUXSEUH7ZFT6E5T2CMQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.590034610 (242h42m27.62960774s) [noise=true , address=81.171.14.70:4027 , piece_id=EN5GYV7RHFSO3VZM2L7R6VU6SHGDASMRW5J6JZTAV3PD7CM5BX2Q, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.590942148 (242h42m27.630515278s) [noise=true , address=81.171.14.70:4036 , piece_id=JVVST4DM4RGMPMA7MCXKEF4CZ6BVMYS2EPKQIOIYSBBQVN2OMURA, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.598615219 (242h42m27.638188339s) [noise=true , address=81.171.14.70:4030 , piece_id=AZ5TFWUXQDXGSI7TS67JS4WNLI3NDS6WGXBLE4RPG3VCTD7C66XQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.605121469 (242h42m27.644694599s) [noise=true , address=81.171.14.70:4028 , piece_id=K56FOSNYNRXVLN4ZALOXWTRSYUDQGEJSE2CXBIWVSO3N2BMIOGNQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.608607507 (242h42m27.648180647s) [noise=true , address=89.149.201.147:4007 , piece_id=XVDNKW23BOGM2ABQQUARBQJL76Z6JXFJOYMV7C2DFBABUYSX53YQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.616551795 (242h42m27.656124925s) [noise=true , address=89.149.201.147:4009 , piece_id=TYWG4XKLX2SUVWCL354TXSCXDMWIUIDLYAB6PC35WH2VD37PCCHA, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:51.629765428 (242h42m27.669338558s) [noise=true , address=89.149.201.147:4011 , piece_id=U5XKHDKNPCIZHO2SSCCPZYOFMGICFX65JIZ4HIPQQZB3OFV6SG5A, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.002319451 (242h42m28.041892571s) [noise=true , address=81.171.14.70:4040 , piece_id=JP3MYMFMO3TAOJU7SSPHOFVLXXY6WVVOTLI5E4LTPVQ77OTMN5XQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.059732871 (242h42m28.099306001s) [noise=true , address=81.171.14.70:4025 , piece_id=QYT6QLDQ2536TIUPYP7UMVNVIK3MX56FHIPMEWOZ73MOX3HXIPBA, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.108620717 (242h42m28.148193847s) [noise=true , address=89.149.201.147:4017 , piece_id=OPDS3GSZDWSF7HA3ZXU6AA4VXLFKFEDVN2AR62Q34TNXNOD5QSXQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.126903312 (242h42m28.166476442s) [noise=true , address=185.167.204.206:4064 , piece_id=ZTH32KU3O7FNIZVSLOCYGAUB4PDTQ6SL437RX4TRGEPZEYDXWKAQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.129735646 (242h42m28.169308776s) [noise=true , address=185.167.204.206:4056 , piece_id=5XZGML67X3A2ZXM2MX7SZF3WQ4JBOKTKJHUF4VXLJDATGPGVBEFA, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.143060059 (242h42m28.182633189s) [noise=true , address=185.167.204.206:4047 , piece_id=RYUCQA53TCVLCV4JVLIFYRCFKMLIJBZDLSTBQNTC4I5JNBSG4KVQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.177316692 (242h42m28.216889822s) [noise=true , address=185.167.204.206:4067 , piece_id=X5K6LFA3QXXXFWZUFU7L5LG57IZJASOGW3XSKWFB5335JANABQKQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.196525832 (242h42m28.236098962s) [noise=true , address=185.167.204.205:4030 , piece_id=TAGCXU46S677M3JO7BIQSH4KNYBQZSZVSZKMNO54ZAVLG5BOXBKQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.229056029 (242h42m28.268629159s) [noise=true , address=185.167.204.206:4055 , piece_id=JZF47O2J6BX5VDTBUZ6U6ER57GN76NKW3G3JZEZWYLQF57RDGAMA, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.229587140 (242h42m28.26916027s) [noise=true , address=185.167.204.206:4038 , piece_id=QWVRZKVI3SDQIVXLAE62OSGVIXXZXQNPJTXG3AYGDZRZRXBJNCCA, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.240029164 (242h42m28.279602294s) [noise=true , address=185.167.204.206:4050 , piece_id=RIXAR37G7AFQZSWIKO2Z5DDNGC43SRJ535HI4N4MLJ4CNXFC76OA, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.250311604 (242h42m28.289884734s) [noise=true , address=185.167.204.206:4049 , piece_id=D6IGZ7FDPPCGYS6EFKFPOX6S5S64MR26TYNOIQT5AGSH55OFLMOQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.260192480 (242h42m28.29976561s) [noise=true , address=185.167.204.206:4060 , piece_id=BUQDUOMBILGOED5KSNACIU7T2ROKDFV2Z34PSQYZ2FWH4TYUTQNQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: <nil> Sep 16 16:10:52.260224291 (242h42m28.299797421s) [seg_pos={0 0}, upload=213252]: Segment reached optimal threshold of 26 pieces. Sep 16 16:10:52.260376130 (242h42m28.29994927s) [noise=true , address=185.167.204.205:4035 , piece_id=GQA5PFXPIOA77UQPZ5NZ5ZO7VKD5C2XKOMIIDZDVYYXUEVVXFCJQ, seg_pos={0 0}, upload=213252]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TT3Cyz1GPrGa9P9jWePFarF4LFaJfyfRbGM1hH5ctGx65JUJp): protocol: expected piece hash; context canceled; EOF Sep 16 16:10:52.260516888 (242h42m28.300090018s) [noise=true , address=185.167.204.205:4028 , piece_id=ZRNLEWOJYLDJJZ6X32ICTDHYCICG4XDWH2KW6Z2RAA5CBEHGVF5A, seg_pos={0 0}, upload=213252]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61): protocol: expected piece hash; context canceled; EOF Sep 16 16:10:52.260535003 (242h42m28.300108133s) [seg_pos={0 0}, upload=213252]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Sep 16 16:10:52.260618952 (242h42m28.300192082s) [upload=213252]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 16 16:10:52.260624764 (242h42m28.300197894s) [upload=213252]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 16 16:10:52.542135655 (242h42m28.581708786s) [upload=213252]: Done uploading segments... Sep 16 16:10:52.542156926 (242h42m28.581730046s) [upload=213252]: Upload finished. err: <nil> Duration: 1m0.002124456s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 14 out of 60 is failed (threshold=3) [........................................XX..X..XXX.XXXXXX.XX] duration: 30.000586531s FAIL (2025-09-16 16:11:43 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.00068595s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 18 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXX..........................................] duration: 6.314030792s FAIL (2025-09-16 16:11:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 36:23.221360752 (242h7m59.260933872s) [seg_pos={0 0}, upload=212758]: Begin upload segment... Sep 16 15:36:23.221427429 (242h7m59.261000559s) [seg_pos={0 0}, upload=212758]: Done begin upload segment. Sep 16 15:36:23.221471934 (242h7m59.261045064s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.221781523 (242h7m59.261354683s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222170084 (242h7m59.261743224s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222380514 (242h7m59.261953644s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222688551 (242h7m59.262261681s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222978833 (242h7m59.262551963s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223291669 (242h7m59.262864809s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223483104 (242h7m59.263056234s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223736246 (242h7m59.263309376s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223952347 (242h7m59.263525477s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224092043 (242h7m59.263665173s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224449704 (242h7m59.264022844s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224537902 (242h7m59.264111032s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224972190 (242h7m59.26454532s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225002056 (242h7m59.264575176s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225500526 (242h7m59.265073646s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225597541 (242h7m59.265170671s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225993644 (242h7m59.265566774s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:24.615369315 (242h8m0.654942445s) [upload=212758]: Next returned nil segment Sep 16 15:36:24.615405684 (242h8m0.654978814s) [upload=212758]: Waiting for error group managing segments... Sep 16 15:36:25.024537642 (242h8m1.064110772s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:07.783420676 (242h8m43.822993807s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:08.809548412 (242h8m44.849121542s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:09.417370950 (242h8m45.45694407s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:10.232853931 (242h8m46.272427062s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.251045936 (242h8m47.290619056s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.530631565 (242h8m47.570204695s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:12.400109127 (242h8m48.439682257s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:13.851060301 (242h8m49.890633431s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.293316228 (242h8m50.332889378s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.648711487 (242h8m50.688284607s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:15.183813908 (242h8m51.223387028s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:19.546944067 (242h8m55.586517197s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:21.938117511 (242h8m57.977690642s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938325488 (242h8m57.977898618s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12EsmL66Gn6K9cUxHqqGtuUZxUGyWCNL4oqyHzS1XUbWVnMMJbd, address:145.79.227.48:4314): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938466356 (242h8m57.978039476s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection Sep 16 15:37:21.938531310 (242h8m57.97810444s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12V9GZR1rr6VfEsK83Bonq2HyceXLcAKnYfoQmA1Qvt5h9u8sik, address:145.79.227.28:4742): protocol: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection Sep 16 15:37:21.938593297 (242h8m57.978166417s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection Sep 16 15:37:21.938613285 (242h8m57.978186405s) [seg_pos={0 0}, upload=212758]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 16 15:37:22.200860966 (242h8m58.240434086s) [upload=212758]: Done uploading segments... Sep 16 15:37:22.200890762 (242h8m58.240463893s) [upload=212758]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.314059617s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 744.183186ms target: https://us1.storj.io/login FAIL (2025-09-16 16:11:34 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 744.216099ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.XXX.X..XX..................................................] duration: 7.057497729s FAIL (2025-09-16 16:11:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.0575352s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 755.289515ms target: https://us1.storj.io/login FAIL (2025-09-16 16:11:10 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 755.321467ms core satellite us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.........................................XX..X..XXX.XXXXXX.X] duration: 30.004180251s FAIL (2025-09-16 16:11:04 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.004211291s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..XXX.X..XX.................................................] duration: 6.271925976s FAIL (2025-09-16 16:10:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.271956935s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 11 out of 60 is failed (threshold=3) [...............................................XX.X.XXXXXXXX] duration: 1m0.000814489s FAIL (2025-09-16 16:10:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Sep 16 16:09:49.760662494 (242h41m25.800235625s) [upload=213239]: Starting upload Sep 16 16:09:49.760731426 (242h41m25.800304556s) [upload=213239]: Done starting upload Sep 16 16:09:49.760949240 (242h41m25.80052237s) [upload=213239]: Uploading segments... Sep 16 16:09:49.760997002 (242h41m25.800570132s) [upload=213239]: Got next segment. Inline: false Sep 16 16:09:49.761025957 (242h41m25.800599077s) [upload=213239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 16:09:49.761034383 (242h41m25.800607503s) [upload=213239]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 16:10:49.283002920 (242h42m25.32257604s) [upload=213239]: Next segment err: context deadline exceeded Sep 16 16:10:49.283243799 (242h42m25.322816929s) [upload=213239]: Done uploading segments... Sep 16 16:10:49.283285999 (242h42m25.322859119s) [upload=213239]: Upload finished. err: context deadline exceeded Duration: 1m0.000849054s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 19 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXX.........................................] duration: 6.471874731s FAIL (2025-09-16 16:10:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 36:23.221360752 (242h7m59.260933872s) [seg_pos={0 0}, upload=212758]: Begin upload segment... Sep 16 15:36:23.221427429 (242h7m59.261000559s) [seg_pos={0 0}, upload=212758]: Done begin upload segment. Sep 16 15:36:23.221471934 (242h7m59.261045064s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.221781523 (242h7m59.261354683s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222170084 (242h7m59.261743224s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222380514 (242h7m59.261953644s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222688551 (242h7m59.262261681s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222978833 (242h7m59.262551963s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223291669 (242h7m59.262864809s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223483104 (242h7m59.263056234s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223736246 (242h7m59.263309376s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223952347 (242h7m59.263525477s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224092043 (242h7m59.263665173s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224449704 (242h7m59.264022844s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224537902 (242h7m59.264111032s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224972190 (242h7m59.26454532s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225002056 (242h7m59.264575176s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225500526 (242h7m59.265073646s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225597541 (242h7m59.265170671s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225993644 (242h7m59.265566774s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:24.615369315 (242h8m0.654942445s) [upload=212758]: Next returned nil segment Sep 16 15:36:24.615405684 (242h8m0.654978814s) [upload=212758]: Waiting for error group managing segments... Sep 16 15:36:25.024537642 (242h8m1.064110772s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:07.783420676 (242h8m43.822993807s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:08.809548412 (242h8m44.849121542s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:09.417370950 (242h8m45.45694407s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:10.232853931 (242h8m46.272427062s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.251045936 (242h8m47.290619056s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.530631565 (242h8m47.570204695s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:12.400109127 (242h8m48.439682257s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:13.851060301 (242h8m49.890633431s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.293316228 (242h8m50.332889378s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.648711487 (242h8m50.688284607s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:15.183813908 (242h8m51.223387028s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:19.546944067 (242h8m55.586517197s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:21.938117511 (242h8m57.977690642s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938325488 (242h8m57.977898618s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12EsmL66Gn6K9cUxHqqGtuUZxUGyWCNL4oqyHzS1XUbWVnMMJbd, address:145.79.227.48:4314): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938466356 (242h8m57.978039476s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection Sep 16 15:37:21.938531310 (242h8m57.97810444s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12V9GZR1rr6VfEsK83Bonq2HyceXLcAKnYfoQmA1Qvt5h9u8sik, address:145.79.227.28:4742): protocol: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection Sep 16 15:37:21.938593297 (242h8m57.978166417s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection Sep 16 15:37:21.938613285 (242h8m57.978186405s) [seg_pos={0 0}, upload=212758]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 16 15:37:22.200860966 (242h8m58.240434086s) [upload=212758]: Done uploading segments... Sep 16 15:37:22.200890762 (242h8m58.240463893s) [upload=212758]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.471922532s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 746.014942ms target: https://us1.storj.io/login FAIL (2025-09-16 16:10:17 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 746.046723ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [X..XXX.X..XX................................................] duration: 7.748707322s FAIL (2025-09-16 16:10:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.748739014s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X..XXX.X..XX...............................................] duration: 6.528221529s FAIL (2025-09-16 16:09:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.528272246s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 10 out of 60 is failed (threshold=3) [................................................XX.X.XXXXXXX] duration: 1m0.002157011s FAIL (2025-09-16 16:09:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== U6FEMYNKGWABBO5JH4AVEFXWK7MJJBRBT6LZRLTF75BEO2Q, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.479723239 (242h40m25.519296369s) [noise=true , address=81.171.14.70:4026 , piece_id=RSOWO25NQZFIRDOH2R6P6CYZM6AHFNVGHQID374OIEPRTYGUND4Q, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.480644023 (242h40m25.520217163s) [noise=true , address=89.149.201.147:4005 , piece_id=BGUJRUESJZE7EUY5YDYETYSFCKGWWQSPUBE5XQKAOHVMRB3VPJNA, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.481191766 (242h40m25.520764886s) [noise=true , address=89.149.201.147:4020 , piece_id=KECW2M7HNXT35F6HBZCUR4XEX3UDC5ZFSL2RMHOPJLWG6CWG4P7A, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.481469184 (242h40m25.521042314s) [noise=true , address=185.167.204.206:4044 , piece_id=R7BMGKJNZ5F7YT3S55MC2NIBOZRR6VIZBMETBHRJW3JGJA6FTG5A, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.482680250 (242h40m25.52225337s) [noise=true , address=81.171.14.70:4044 , piece_id=X6UTP5E42VDYPE4JXTKH6HW7EPWTT53ADOVKQ3CMOTGUQWE33MQQ, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.483524568 (242h40m25.523097698s) [noise=true , address=89.149.201.147:4007 , piece_id=6V7CYY5EJFWPYD43J2O7NYXFOANECOWLH3QTEWNBIV3EHAZH2X5A, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.484677874 (242h40m25.524251004s) [noise=true , address=89.149.201.147:4016 , piece_id=XD2XH33A4NRKKDTZLIW57V25B5WHSGZFZYPGHUYHZO2VH3DJN2YQ, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.485118924 (242h40m25.524692044s) [noise=true , address=89.149.201.147:4006 , piece_id=6UX57OQRKRIJXXITS3ZO2DNCXDOHDKKPLTTUXXYWVTS277K3EXYA, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.485507744 (242h40m25.525080875s) [noise=true , address=185.167.204.205:4025 , piece_id=6YMFYBYGSDWLR4K5FCU5LAOZ5S7J2TC55NK2F4JU75P3DVQ3P5SA, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.485909921 (242h40m25.525483051s) [noise=true , address=185.167.204.206:4057 , piece_id=YUDKXRX7S5HFPYPI2ON4SILBMDXOWHPSB4G334W7RWBROR54LLDQ, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.486786349 (242h40m25.52635948s) [noise=true , address=185.167.204.205:4016 , piece_id=BZOTKSQOQJZ6UU5DXR2GHCFXMI6OQREUD6JOOXXZ6BPND62EPUBA, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:49.490311582 (242h40m25.529884712s) [noise=true , address=185.167.204.205:4002 , piece_id=3SS2MJZDSS23M7LLQTSPUSUOUXJ5NXHT4YX66PJDDIW3DR7NOU6A, seg_pos={0 0}, upload=213225]: Uploading piece... Sep 16 16:08:50.328387314 (242h40m26.367960444s) [upload=213225]: Next returned nil segment Sep 16 16:08:50.328409998 (242h40m26.367983118s) [upload=213225]: Waiting for error group managing segments... Sep 16 16:08:50.709035331 (242h40m26.748608461s) [noise=true , address=81.171.14.70:4043 , piece_id=E36M5TVVESAFZVSTJXUCHPOP3LG5ORADMC4H7BGNUO47TSVIBENQ, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.712373307 (242h40m26.751946437s) [noise=true , address=81.171.14.70:4026 , piece_id=RSOWO25NQZFIRDOH2R6P6CYZM6AHFNVGHQID374OIEPRTYGUND4Q, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.718171829 (242h40m26.757744949s) [noise=true , address=81.171.14.70:4028 , piece_id=J7ODAVCUFWPSX5D6AFJMAYS75GYTABJQUF6GKB3TYKKRCFF3BGQQ, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.744119370 (242h40m26.78369249s) [noise=true , address=89.149.201.147:4020 , piece_id=KECW2M7HNXT35F6HBZCUR4XEX3UDC5ZFSL2RMHOPJLWG6CWG4P7A, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.747449933 (242h40m26.787023053s) [noise=true , address=81.171.14.70:4041 , piece_id=BXPPI6FY3ZRHC6RCXVPPZLAA53IIGG4GNSTDQDKQKNE573RDV62Q, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.751439410 (242h40m26.79101253s) [noise=true , address=89.149.201.147:4022 , piece_id=FGMVYMZ5CNRL2K5AXZ5A57TUNVCVCBHDZSTFUY55ODGDJCHIR33Q, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.758623570 (242h40m26.7981967s) [noise=true , address=81.171.14.70:4027 , piece_id=B6DICW6OBXGVUTMUEQE6NX2JQ3P37MD65YEEN323V2S2EXF2A3XQ, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.758651724 (242h40m26.798224854s) [noise=true , address=81.171.14.70:4042 , piece_id=OXT2YW4XLXWLYXMPP7ZOP4UBO6VVOZCVFTLC3UPH5ZYNZCVHTETA, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.760195213 (242h40m26.799768343s) [noise=true , address=89.149.201.147:4006 , piece_id=6UX57OQRKRIJXXITS3ZO2DNCXDOHDKKPLTTUXXYWVTS277K3EXYA, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.778561176 (242h40m26.818134296s) [noise=true , address=89.149.201.147:4005 , piece_id=BGUJRUESJZE7EUY5YDYETYSFCKGWWQSPUBE5XQKAOHVMRB3VPJNA, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.797232462 (242h40m26.836805582s) [noise=true , address=89.149.201.147:4024 , piece_id=DPBZ7U6FEMYNKGWABBO5JH4AVEFXWK7MJJBRBT6LZRLTF75BEO2Q, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.798421947 (242h40m26.837995087s) [noise=true , address=89.149.201.147:4019 , piece_id=RLVMS7MKM6N6LCPOZ7OIRKS5AJQA2W6KRZL74BUVCBNZAYT3VBVA, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.823262901 (242h40m26.862836031s) [noise=true , address=89.149.201.147:4016 , piece_id=XD2XH33A4NRKKDTZLIW57V25B5WHSGZFZYPGHUYHZO2VH3DJN2YQ, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:50.831862126 (242h40m26.871435266s) [noise=true , address=89.149.201.147:4014 , piece_id=VYPUNKZ57ONN6HZWKTBGQ7K4NUY7HDWEY2BAZSPNTTKGYLY6E6IQ, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.166819445 (242h40m27.206392575s) [noise=true , address=81.171.14.70:4047 , piece_id=NWLXMDKX3JJXZPDDT3K2X6ZYA6BH3IB3YMQS7UPUYHPOD25DCHQA, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.193680766 (242h40m27.233253896s) [noise=true , address=81.171.14.70:4036 , piece_id=A362QBSIY5L5MSDVBKYMVHV6KORX7RUYTVSJUUONDC7K2QHHRR6A, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.218249391 (242h40m27.257822521s) [noise=true , address=81.171.14.70:4044 , piece_id=X6UTP5E42VDYPE4JXTKH6HW7EPWTT53ADOVKQ3CMOTGUQWE33MQQ, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.234282833 (242h40m27.273855963s) [noise=true , address=81.171.14.70:4032 , piece_id=XCS3O5YFC3IDIJO4UHI7JSNGONYNRQGIZCKDNSVEBMTP5JUD4O2Q, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.291575744 (242h40m27.331148884s) [noise=true , address=185.167.204.206:4049 , piece_id=JNGXSZTJKVZEDBCG7SMD65RLWA6TNYKBVHMMX5OLTIPXYVAKWXAA, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.307316689 (242h40m27.346889819s) [noise=true , address=89.149.201.147:4007 , piece_id=6V7CYY5EJFWPYD43J2O7NYXFOANECOWLH3QTEWNBIV3EHAZH2X5A, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.348584564 (242h40m27.388157694s) [noise=true , address=185.167.204.206:4044 , piece_id=R7BMGKJNZ5F7YT3S55MC2NIBOZRR6VIZBMETBHRJW3JGJA6FTG5A, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.365197019 (242h40m27.404770149s) [noise=true , address=185.167.204.205:4002 , piece_id=3SS2MJZDSS23M7LLQTSPUSUOUXJ5NXHT4YX66PJDDIW3DR7NOU6A, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.367782583 (242h40m27.407355713s) [noise=true , address=185.167.204.206:4068 , piece_id=UM4W4LQ5BCBJMRNZ4DXYUIVIHNCHR6REMYZ62YO2H3WRVW3VBAJQ, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.378761409 (242h40m27.418334539s) [noise=true , address=185.167.204.205:4025 , piece_id=6YMFYBYGSDWLR4K5FCU5LAOZ5S7J2TC55NK2F4JU75P3DVQ3P5SA, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.407961694 (242h40m27.447534825s) [noise=true , address=185.167.204.205:4012 , piece_id=5QTLW3YR26YJPRS7UG65PPR5HKMSDYETCPYYDNE5TFPZ3UVAVGCQ, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.410211148 (242h40m27.449784268s) [noise=true , address=185.167.204.205:4016 , piece_id=BZOTKSQOQJZ6UU5DXR2GHCFXMI6OQREUD6JOOXXZ6BPND62EPUBA, seg_pos={0 0}, upload=213225]: Done uploading piece. err: <nil> Sep 16 16:08:51.410238570 (242h40m27.4498117s) [seg_pos={0 0}, upload=213225]: Segment reached optimal threshold of 26 pieces. Sep 16 16:08:51.410421950 (242h40m27.44999507s) [noise=true , address=185.167.204.206:4046 , piece_id=KDHOHE3QRGXHIUGCUIXJKTIIZRA5ZCHLWCQUW2IFK2FQMVZBBV5A, seg_pos={0 0}, upload=213225]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5): protocol: expected piece hash; context canceled; EOF Sep 16 16:08:51.410523273 (242h40m27.450096393s) [noise=true , address=185.167.204.206:4057 , piece_id=YUDKXRX7S5HFPYPI2ON4SILBMDXOWHPSB4G334W7RWBROR54LLDQ, seg_pos={0 0}, upload=213225]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF Sep 16 16:08:51.410572346 (242h40m27.450145466s) [seg_pos={0 0}, upload=213225]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Sep 16 16:08:51.410659962 (242h40m27.450233092s) [upload=213225]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 16 16:08:51.410664942 (242h40m27.450238072s) [upload=213225]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 16 16:08:51.682131358 (242h40m27.721704488s) [upload=213225]: Done uploading segments... Sep 16 16:08:51.682156625 (242h40m27.721729755s) [upload=213225]: Upload finished. err: <nil> Duration: 1m0.002177249s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 20 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXX........................................] duration: 6.601031748s FAIL (2025-09-16 16:09:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 36:23.221360752 (242h7m59.260933872s) [seg_pos={0 0}, upload=212758]: Begin upload segment... Sep 16 15:36:23.221427429 (242h7m59.261000559s) [seg_pos={0 0}, upload=212758]: Done begin upload segment. Sep 16 15:36:23.221471934 (242h7m59.261045064s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.221781523 (242h7m59.261354683s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222170084 (242h7m59.261743224s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222380514 (242h7m59.261953644s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222688551 (242h7m59.262261681s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222978833 (242h7m59.262551963s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223291669 (242h7m59.262864809s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223483104 (242h7m59.263056234s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223736246 (242h7m59.263309376s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223952347 (242h7m59.263525477s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224092043 (242h7m59.263665173s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224449704 (242h7m59.264022844s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224537902 (242h7m59.264111032s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224972190 (242h7m59.26454532s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225002056 (242h7m59.264575176s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225500526 (242h7m59.265073646s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225597541 (242h7m59.265170671s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225993644 (242h7m59.265566774s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:24.615369315 (242h8m0.654942445s) [upload=212758]: Next returned nil segment Sep 16 15:36:24.615405684 (242h8m0.654978814s) [upload=212758]: Waiting for error group managing segments... Sep 16 15:36:25.024537642 (242h8m1.064110772s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:07.783420676 (242h8m43.822993807s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:08.809548412 (242h8m44.849121542s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:09.417370950 (242h8m45.45694407s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:10.232853931 (242h8m46.272427062s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.251045936 (242h8m47.290619056s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.530631565 (242h8m47.570204695s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:12.400109127 (242h8m48.439682257s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:13.851060301 (242h8m49.890633431s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.293316228 (242h8m50.332889378s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.648711487 (242h8m50.688284607s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:15.183813908 (242h8m51.223387028s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:19.546944067 (242h8m55.586517197s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:21.938117511 (242h8m57.977690642s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938325488 (242h8m57.977898618s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12EsmL66Gn6K9cUxHqqGtuUZxUGyWCNL4oqyHzS1XUbWVnMMJbd, address:145.79.227.48:4314): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938466356 (242h8m57.978039476s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection Sep 16 15:37:21.938531310 (242h8m57.97810444s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12V9GZR1rr6VfEsK83Bonq2HyceXLcAKnYfoQmA1Qvt5h9u8sik, address:145.79.227.28:4742): protocol: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection Sep 16 15:37:21.938593297 (242h8m57.978166417s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection Sep 16 15:37:21.938613285 (242h8m57.978186405s) [seg_pos={0 0}, upload=212758]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 16 15:37:22.200860966 (242h8m58.240434086s) [upload=212758]: Done uploading segments... Sep 16 15:37:22.200890762 (242h8m58.240463893s) [upload=212758]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.601067166s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..........................................XX..X..XXX.XXXXXX.] duration: 5.586916743s FAIL (2025-09-16 16:09:26 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.586957029s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...........................................XX..X..XXX.XXXXXX] duration: 30.004452963s FAIL (2025-09-16 16:09:17 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.004483931s core uplink eu1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 763.116268ms target: https://us1.storj.io/login FAIL (2025-09-16 16:09:07 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 763.158408ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..X..XXX.X..XX..............................................] duration: 6.250277523s FAIL (2025-09-16 16:09:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.250344209s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...X..XXX.X..XX.............................................] duration: 10.071853296s FAIL (2025-09-16 16:08:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 10.071892971s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 9 out of 60 is failed (threshold=3) [.................................................XX.X.XXXXXX] duration: 1m0.001551806s FAIL (2025-09-16 16:08:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Sep 16 16:07:43.098117560 (242h39m19.13769069s) [upload=213211]: Starting upload Sep 16 16:07:43.098228982 (242h39m19.137802112s) [upload=213211]: Done starting upload Sep 16 16:07:43.098300548 (242h39m19.137873678s) [upload=213211]: Uploading segments... Sep 16 16:07:43.098479268 (242h39m19.138052408s) [upload=213211]: Got next segment. Inline: false Sep 16 16:07:43.098524385 (242h39m19.138097515s) [upload=213211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 16:07:43.098532149 (242h39m19.13810527s) [upload=213211]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 16:08:42.617836074 (242h40m18.657409204s) [upload=213211]: Next segment err: context deadline exceeded Sep 16 16:08:42.618011639 (242h40m18.657584769s) [upload=213211]: Done uploading segments... Sep 16 16:08:42.618037678 (242h40m18.657610808s) [upload=213211]: Upload finished. err: context deadline exceeded Duration: 1m0.001569911s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 21 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXX.......................................] duration: 6.753734923s FAIL (2025-09-16 16:08:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 36:23.221360752 (242h7m59.260933872s) [seg_pos={0 0}, upload=212758]: Begin upload segment... Sep 16 15:36:23.221427429 (242h7m59.261000559s) [seg_pos={0 0}, upload=212758]: Done begin upload segment. Sep 16 15:36:23.221471934 (242h7m59.261045064s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.221781523 (242h7m59.261354683s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222170084 (242h7m59.261743224s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222380514 (242h7m59.261953644s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222688551 (242h7m59.262261681s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222978833 (242h7m59.262551963s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223291669 (242h7m59.262864809s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223483104 (242h7m59.263056234s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223736246 (242h7m59.263309376s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223952347 (242h7m59.263525477s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224092043 (242h7m59.263665173s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224449704 (242h7m59.264022844s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224537902 (242h7m59.264111032s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224972190 (242h7m59.26454532s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225002056 (242h7m59.264575176s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225500526 (242h7m59.265073646s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225597541 (242h7m59.265170671s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225993644 (242h7m59.265566774s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:24.615369315 (242h8m0.654942445s) [upload=212758]: Next returned nil segment Sep 16 15:36:24.615405684 (242h8m0.654978814s) [upload=212758]: Waiting for error group managing segments... Sep 16 15:36:25.024537642 (242h8m1.064110772s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:07.783420676 (242h8m43.822993807s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:08.809548412 (242h8m44.849121542s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:09.417370950 (242h8m45.45694407s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:10.232853931 (242h8m46.272427062s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.251045936 (242h8m47.290619056s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.530631565 (242h8m47.570204695s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:12.400109127 (242h8m48.439682257s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:13.851060301 (242h8m49.890633431s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.293316228 (242h8m50.332889378s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.648711487 (242h8m50.688284607s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:15.183813908 (242h8m51.223387028s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:19.546944067 (242h8m55.586517197s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:21.938117511 (242h8m57.977690642s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938325488 (242h8m57.977898618s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12EsmL66Gn6K9cUxHqqGtuUZxUGyWCNL4oqyHzS1XUbWVnMMJbd, address:145.79.227.48:4314): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938466356 (242h8m57.978039476s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection Sep 16 15:37:21.938531310 (242h8m57.97810444s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12V9GZR1rr6VfEsK83Bonq2HyceXLcAKnYfoQmA1Qvt5h9u8sik, address:145.79.227.28:4742): protocol: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection Sep 16 15:37:21.938593297 (242h8m57.978166417s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection Sep 16 15:37:21.938613285 (242h8m57.978186405s) [seg_pos={0 0}, upload=212758]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 16 15:37:22.200860966 (242h8m58.240434086s) [upload=212758]: Done uploading segments... Sep 16 15:37:22.200890762 (242h8m58.240463893s) [upload=212758]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.753783193s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 784.923486ms target: https://us1.storj.io/login FAIL (2025-09-16 16:08:38 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 784.954083ms core satellite us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 11 out of 60 is failed (threshold=3) [............................................XX..X..XXX.XXXXX] duration: 30.004989682s FAIL (2025-09-16 16:08:16 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.005013046s core uplink eu1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 699.633261ms target: https://us1.storj.io/login FAIL (2025-09-16 16:07:55 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 699.698605ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....X..XXX.X..XX............................................] duration: 6.749662946s FAIL (2025-09-16 16:07:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.749690668s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 8 out of 60 is failed (threshold=3) [..................................................XX.X.XXXXX] duration: 1m0.003073303s FAIL (2025-09-16 16:07:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HPG2X3B65EGPG5HTZRGJ47EFYRVHYQD2GNSXUGHHQ, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.167205672 (242h38m19.206778792s) [noise=true , address=89.149.201.147:4016 , piece_id=PZVCFUK3JBFIOFMAUW75PH25VZSTLJVWILNADC7CU4FVRJHSZXUQ, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.167234806 (242h38m19.206807936s) [noise=true , address=185.167.204.205:4031 , piece_id=HILDHRNJWPORMN4D26EYYYCJVRMJP3UQPLM3N4LHJUDD3NMRK6AA, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.167551009 (242h38m19.20712414s) [noise=true , address=81.171.14.70:4046 , piece_id=IOQJW2GRZZSS2F4CHN3DOIC63QWNIH7FRRRRMX2VMZQ772AUARPQ, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.167640569 (242h38m19.207213699s) [noise=true , address=185.167.204.205:4034 , piece_id=5ODKTAFEF4PTZP2BRWPXRPT2ICHEK2KJLDAJ6RSMTW3RIPXUUZ4Q, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.167888151 (242h38m19.207461281s) [noise=true , address=185.167.204.206:4041 , piece_id=7ICAZ2IJOOK5YBLVHRHTZMQQ4AEKY5Z7C7VC2GZ5WYFIRI3R4PJQ, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.168022226 (242h38m19.207595356s) [noise=true , address=81.171.14.70:4040 , piece_id=TJSRVYVMILAAEAGKN65KCWG3QEIN2WYFUXHVX4FIXA2YYX3ILZRQ, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.168132957 (242h38m19.207706077s) [noise=true , address=185.167.204.206:4056 , piece_id=TYNLEFBOZW6CMEELPTUNTMANJ2TVPF3LQCVBIIWHYYBW44LRTE2Q, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.168459880 (242h38m19.20803301s) [noise=true , address=89.149.201.147:4008 , piece_id=DDLO2UM24Z5IPH3GKCXXSW4T4ZTMI7VE7FULRA2GEK6LF5UGK7TA, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.168548608 (242h38m19.208121738s) [noise=true , address=89.149.201.147:4011 , piece_id=YB2INZ73HTCWBNMETJP5SN3PLKZ6Z5WOAO7RQH3IXQMA6IEJV6LA, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.168900398 (242h38m19.208473528s) [noise=true , address=185.167.204.206:4047 , piece_id=LZUWBZLVBDWBWXBZ6SLBRCU32GPAFQZT4PZ7T7PYAQDTX7O7CNXQ, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.168956796 (242h38m19.208529926s) [noise=true , address=185.167.204.206:4064 , piece_id=4UMKSZK4FVCFTAF3D76AEKHHP6R5KU5TSYEUYMNASZXRXQJBVQ4Q, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:43.169506272 (242h38m19.209079403s) [noise=true , address=185.167.204.206:4071 , piece_id=UVZC4CP4AWXUPSYN3T7UZ2DZMRWJOAY5EQJGUUGW46VIDGKYF5TA, seg_pos={0 0}, upload=213199]: Uploading piece... Sep 16 16:06:44.001367730 (242h38m20.040940891s) [upload=213199]: Next returned nil segment Sep 16 16:06:44.001393870 (242h38m20.040967001s) [upload=213199]: Waiting for error group managing segments... Sep 16 16:06:44.373229916 (242h38m20.412803086s) [noise=true , address=81.171.14.70:4040 , piece_id=TJSRVYVMILAAEAGKN65KCWG3QEIN2WYFUXHVX4FIXA2YYX3ILZRQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.380402865 (242h38m20.419975995s) [noise=true , address=81.171.14.70:4035 , piece_id=IRG7VHZA6NPHPG2X3B65EGPG5HTZRGJ47EFYRVHYQD2GNSXUGHHQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.406572269 (242h38m20.446145399s) [noise=true , address=81.171.14.70:4033 , piece_id=AYQY462MLV5OQLQ5GQJ6SPK4OIG5OHMRRV5AQ2RFVFSFB5R5T77A, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.422040985 (242h38m20.461614105s) [noise=true , address=81.171.14.70:4027 , piece_id=4TD64M767PSXKLEJLGNOI2KT3ZCJJDGBDZU3UF6IMJ36YAXGVTZQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.425231040 (242h38m20.46480417s) [noise=true , address=81.171.14.70:4041 , piece_id=XYRZ37HGHGUCF52CP2JPH3ICTWWMFD4Z34NY3CNHB43JA2H3SPZQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.442214642 (242h38m20.481787772s) [noise=true , address=81.171.14.70:4048 , piece_id=DBZDSPPSHGWBUATZ3QFNNN4Q6ESFX7E4GPIOS3M7ODICJ6O2GZVQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.444794935 (242h38m20.484368055s) [noise=true , address=81.171.14.70:4045 , piece_id=CKN4BRDVBWGSWCUKWWR5IKFX7GSSOKTNBBZHTZ7ZD6ZCOKJVNQFA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.465529960 (242h38m20.505103091s) [noise=true , address=89.149.201.147:4016 , piece_id=PZVCFUK3JBFIOFMAUW75PH25VZSTLJVWILNADC7CU4FVRJHSZXUQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.468117327 (242h38m20.507690447s) [noise=true , address=89.149.201.147:4011 , piece_id=YB2INZ73HTCWBNMETJP5SN3PLKZ6Z5WOAO7RQH3IXQMA6IEJV6LA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.472395164 (242h38m20.511968284s) [noise=true , address=81.171.14.70:4028 , piece_id=XYXDNVPYPCFWCLYGBGSBEQKBAHK6P3TA7LG3VEHW4TR62IGUKO7Q, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.845309613 (242h38m20.884882733s) [noise=true , address=81.171.14.70:4026 , piece_id=VCYSPYNANVPAQOYWQIV5LQPMO6UPWN3SPNFLROBYP35EG5WNBDMA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.886306071 (242h38m20.925879191s) [noise=true , address=81.171.14.70:4044 , piece_id=VRHLPK7AW4ICNMSCFJLUHL5SMK2LWC2YMXJROCAICCKDYRNLL3BA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.907180202 (242h38m20.946753332s) [noise=true , address=81.171.14.70:4036 , piece_id=ALT5MW5GQUY6SOMDM2FSK33UYGFQL3WQX6P4TMF4TDKQKZLKYUYA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.913834714 (242h38m20.953407854s) [noise=true , address=89.149.201.147:4001 , piece_id=M7KRGNDCY5WBIPOCTQWBQCVBCK2VFAAHMTWH2KCM7YVRQFW73AGQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.950817718 (242h38m20.990390838s) [noise=true , address=89.149.201.147:4008 , piece_id=DDLO2UM24Z5IPH3GKCXXSW4T4ZTMI7VE7FULRA2GEK6LF5UGK7TA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.952108427 (242h38m20.991681547s) [noise=true , address=81.171.14.70:4046 , piece_id=IOQJW2GRZZSS2F4CHN3DOIC63QWNIH7FRRRRMX2VMZQ772AUARPQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:44.991975334 (242h38m21.031548474s) [noise=true , address=185.167.204.206:4047 , piece_id=LZUWBZLVBDWBWXBZ6SLBRCU32GPAFQZT4PZ7T7PYAQDTX7O7CNXQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:45.018435080 (242h38m21.05800821s) [noise=true , address=185.167.204.205:4034 , piece_id=5ODKTAFEF4PTZP2BRWPXRPT2ICHEK2KJLDAJ6RSMTW3RIPXUUZ4Q, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:45.021417028 (242h38m21.060990149s) [noise=true , address=185.167.204.206:4056 , piece_id=TYNLEFBOZW6CMEELPTUNTMANJ2TVPF3LQCVBIIWHYYBW44LRTE2Q, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:45.026911743 (242h38m21.066484863s) [noise=true , address=185.167.204.206:4041 , piece_id=7ICAZ2IJOOK5YBLVHRHTZMQQ4AEKY5Z7C7VC2GZ5WYFIRI3R4PJQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:45.031656218 (242h38m21.071229338s) [noise=true , address=185.167.204.206:4049 , piece_id=DVCK7AQXG6AIEVS5OWTL5G3XGLS3PR4XTKUA5E7BOM2FZLMBSJEA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:45.036658984 (242h38m21.076232104s) [noise=true , address=185.167.204.206:4063 , piece_id=LVPFAC7AB3OQBKP6I3HH4NS5MFFMM5OYI55M3GZ3TAIBMPVB5BOQ, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:45.060995757 (242h38m21.100568887s) [noise=true , address=185.167.204.206:4064 , piece_id=4UMKSZK4FVCFTAF3D76AEKHHP6R5KU5TSYEUYMNASZXRXQJBVQ4Q, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:45.070767906 (242h38m21.110341036s) [noise=true , address=185.167.204.206:4071 , piece_id=UVZC4CP4AWXUPSYN3T7UZ2DZMRWJOAY5EQJGUUGW46VIDGKYF5TA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:45.072677032 (242h38m21.112250152s) [noise=true , address=185.167.204.205:4031 , piece_id=HILDHRNJWPORMN4D26EYYYCJVRMJP3UQPLM3N4LHJUDD3NMRK6AA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:45.076292917 (242h38m21.115866047s) [noise=true , address=185.167.204.206:4070 , piece_id=2BAXGQEMMTAN2MXMNYTBJ7252KJR2UGC66DW6QSNBMX77WQ7P3NA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: <nil> Sep 16 16:06:45.076333324 (242h38m21.115906444s) [seg_pos={0 0}, upload=213199]: Segment reached optimal threshold of 26 pieces. Sep 16 16:06:45.076564655 (242h38m21.116137785s) [noise=true , address=185.167.204.205:4005 , piece_id=7UO6OPNCKV7Z7TEYNAUTSY3NNPQESPB2CVE2EEGOVRXRE4YZD2QA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM): protocol: expected piece hash; context canceled; EOF Sep 16 16:06:45.076677059 (242h38m21.116250179s) [noise=true , address=185.167.204.205:4015 , piece_id=GGTJ24WI2K7L3EDCUUNIZWFA3HUY7F55OH2CCXVYEC26VY6HKYNA, seg_pos={0 0}, upload=213199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pAUTtyVrPw8FCi8oTE1dsU8avWdehnDdCdtYN4P3zedYoBfiX): protocol: expected piece hash; context canceled; EOF Sep 16 16:06:45.076696416 (242h38m21.116269546s) [seg_pos={0 0}, upload=213199]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Sep 16 16:06:45.076782139 (242h38m21.116355259s) [upload=213199]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 16 16:06:45.076789222 (242h38m21.116362342s) [upload=213199]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 16 16:06:45.364784483 (242h38m21.404357614s) [upload=213199]: Done uploading segments... Sep 16 16:06:45.364810353 (242h38m21.404383483s) [upload=213199]: Upload finished. err: <nil> Duration: 1m0.003088301s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXX......................................] duration: 6.32559362s FAIL (2025-09-16 16:07:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 36:23.221360752 (242h7m59.260933872s) [seg_pos={0 0}, upload=212758]: Begin upload segment... Sep 16 15:36:23.221427429 (242h7m59.261000559s) [seg_pos={0 0}, upload=212758]: Done begin upload segment. Sep 16 15:36:23.221471934 (242h7m59.261045064s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.221781523 (242h7m59.261354683s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222170084 (242h7m59.261743224s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222380514 (242h7m59.261953644s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222688551 (242h7m59.262261681s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222978833 (242h7m59.262551963s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223291669 (242h7m59.262864809s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223483104 (242h7m59.263056234s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223736246 (242h7m59.263309376s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223952347 (242h7m59.263525477s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224092043 (242h7m59.263665173s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224449704 (242h7m59.264022844s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224537902 (242h7m59.264111032s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224972190 (242h7m59.26454532s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225002056 (242h7m59.264575176s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225500526 (242h7m59.265073646s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225597541 (242h7m59.265170671s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225993644 (242h7m59.265566774s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:24.615369315 (242h8m0.654942445s) [upload=212758]: Next returned nil segment Sep 16 15:36:24.615405684 (242h8m0.654978814s) [upload=212758]: Waiting for error group managing segments... Sep 16 15:36:25.024537642 (242h8m1.064110772s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:07.783420676 (242h8m43.822993807s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:08.809548412 (242h8m44.849121542s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:09.417370950 (242h8m45.45694407s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:10.232853931 (242h8m46.272427062s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.251045936 (242h8m47.290619056s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.530631565 (242h8m47.570204695s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:12.400109127 (242h8m48.439682257s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:13.851060301 (242h8m49.890633431s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.293316228 (242h8m50.332889378s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.648711487 (242h8m50.688284607s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:15.183813908 (242h8m51.223387028s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:19.546944067 (242h8m55.586517197s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:21.938117511 (242h8m57.977690642s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938325488 (242h8m57.977898618s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12EsmL66Gn6K9cUxHqqGtuUZxUGyWCNL4oqyHzS1XUbWVnMMJbd, address:145.79.227.48:4314): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938466356 (242h8m57.978039476s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection Sep 16 15:37:21.938531310 (242h8m57.97810444s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12V9GZR1rr6VfEsK83Bonq2HyceXLcAKnYfoQmA1Qvt5h9u8sik, address:145.79.227.28:4742): protocol: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection Sep 16 15:37:21.938593297 (242h8m57.978166417s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection Sep 16 15:37:21.938613285 (242h8m57.978186405s) [seg_pos={0 0}, upload=212758]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 16 15:37:22.200860966 (242h8m58.240434086s) [upload=212758]: Done uploading segments... Sep 16 15:37:22.200890762 (242h8m58.240463893s) [upload=212758]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.325623977s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.............................................XX..X..XXX.XXXX] duration: 30.000715166s FAIL (2025-09-16 16:07:07 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000731579s core uplink eu1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 687.871272ms target: https://us1.storj.io/login FAIL (2025-09-16 16:07:02 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 687.899426ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X....X..XXX.X..XX...........................................] duration: 6.380879654s FAIL (2025-09-16 16:06:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.380911474s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 7 out of 60 is failed (threshold=3) [...................................................XX.X.XXXX] duration: 1m0.004236228s FAIL (2025-09-16 16:06:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== O7RA44SSUARNIXYJ64O4PD4JOMEVFSI42R5UUWA, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.917672051 (242h37m18.957245171s) [noise=true , address=81.171.14.70:4025 , piece_id=VEAEO2JXXBQNVBPQBPQE7HEXB7CSG2K3LTWWH26BWLJ24QJNVSBQ, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.917967132 (242h37m18.957540252s) [noise=true , address=185.167.204.205:4031 , piece_id=UPNIRLIUHAJCTRKWHXGHGQFFNPAY3O37BBU7JWNYZXK3FFHOEN4Q, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.918218250 (242h37m18.95779138s) [noise=true , address=81.171.14.70:4036 , piece_id=II26FT7KGTGBI75XUCQMJQDF5YTYNTNSUHAAPCNFGAYQNENGH6MA, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.918495468 (242h37m18.958068588s) [noise=true , address=89.149.201.147:4020 , piece_id=SD77D5IWBHTBKPMR65AJ2OV3J5N762EKZIE7IQMLSJDTEDVZK2QQ, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.918892535 (242h37m18.958465665s) [noise=true , address=89.149.201.147:4006 , piece_id=BQWUGZR7MOAPRCP7CZO3BN4CXNHOE6JIE4Y37YT25X262HXCBMOQ, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.919226982 (242h37m18.958800112s) [noise=true , address=81.171.14.70:4044 , piece_id=UGEW3FK5MJDMO55ATPEWOFCGMALGU7FLJ2VY2F3QNRQPZQMLAKZA, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.919500893 (242h37m18.959074013s) [noise=true , address=185.167.204.205:4005 , piece_id=KKKFBGPIG2HU43SQFXCFZAVFXLO5PVG63DPFRMFMITANZCKA7ATA, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.919812065 (242h37m18.959385185s) [noise=true , address=185.167.204.206:4060 , piece_id=3DD3XYUFZJ4OHPHGI7ZZIGRVXYQM2LN3RIWHT733BCROZYK57NJQ, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.920130683 (242h37m18.959703813s) [noise=true , address=185.167.204.206:4051 , piece_id=PGXE5K6LS3BU72ONIUDPBIWQGLBLCW2ZIRC4RIHAIPB5TQ4BVHXA, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.920569127 (242h37m18.960142257s) [noise=true , address=81.171.14.70:4029 , piece_id=MDME6BHR6LM6LDJNLJRZIVJCRIWFSXGJTBU7DIBSODU6U2IQCOMA, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.920876062 (242h37m18.960449192s) [noise=true , address=89.149.201.147:4005 , piece_id=PQOZP6OXGZNQLNYNCGN2D7U7YRMWI6HOMRKSTM3QW3Y7XPWGO4TA, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:42.921251567 (242h37m18.960824697s) [noise=true , address=81.171.14.70:4030 , piece_id=4DGF6UZMTJ5TRZSFB7MMCNK2KQNQPC2JMYGJUNV6RWC7MFV2VI7A, seg_pos={0 0}, upload=213182]: Uploading piece... Sep 16 16:05:43.663256685 (242h37m19.702829815s) [upload=213182]: Next returned nil segment Sep 16 16:05:43.663278236 (242h37m19.702851366s) [upload=213182]: Waiting for error group managing segments... Sep 16 16:05:44.139426431 (242h37m20.178999551s) [noise=true , address=81.171.14.70:4044 , piece_id=UGEW3FK5MJDMO55ATPEWOFCGMALGU7FLJ2VY2F3QNRQPZQMLAKZA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.152824453 (242h37m20.192397583s) [noise=true , address=81.171.14.70:4028 , piece_id=6TYXUOXYRICKBO7RA44SSUARNIXYJ64O4PD4JOMEVFSI42R5UUWA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.159753478 (242h37m20.199326608s) [noise=true , address=81.171.14.70:4036 , piece_id=II26FT7KGTGBI75XUCQMJQDF5YTYNTNSUHAAPCNFGAYQNENGH6MA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.164645724 (242h37m20.204218854s) [noise=true , address=81.171.14.70:4030 , piece_id=4DGF6UZMTJ5TRZSFB7MMCNK2KQNQPC2JMYGJUNV6RWC7MFV2VI7A, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.165565977 (242h37m20.205139097s) [noise=true , address=81.171.14.70:4026 , piece_id=TGWQELXZM2NC3OAVUCP4EZJ7KONX4Z2CHCOPZFGQV3EDCZ6WGXXA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.171244450 (242h37m20.21081757s) [noise=true , address=81.171.14.70:4032 , piece_id=JZK43KF2UJNEPUBZCGK4NPPF2QM2XUXOTCUJSAXP7NFLD5SMTBGQ, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.171960063 (242h37m20.211533183s) [noise=true , address=81.171.14.70:4029 , piece_id=MDME6BHR6LM6LDJNLJRZIVJCRIWFSXGJTBU7DIBSODU6U2IQCOMA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.177148003 (242h37m20.216721123s) [noise=true , address=81.171.14.70:4025 , piece_id=VEAEO2JXXBQNVBPQBPQE7HEXB7CSG2K3LTWWH26BWLJ24QJNVSBQ, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.191086565 (242h37m20.230659695s) [noise=true , address=89.149.201.147:4005 , piece_id=PQOZP6OXGZNQLNYNCGN2D7U7YRMWI6HOMRKSTM3QW3Y7XPWGO4TA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.202004215 (242h37m20.241577345s) [noise=true , address=89.149.201.147:4024 , piece_id=DOLXMMS4B4DKVKLLJVPWATAQSHTM5343VYNYSS3TD4NXWPHTJHLA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.226511714 (242h37m20.266084834s) [noise=true , address=89.149.201.147:4020 , piece_id=SD77D5IWBHTBKPMR65AJ2OV3J5N762EKZIE7IQMLSJDTEDVZK2QQ, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.256866006 (242h37m20.296439126s) [noise=true , address=89.149.201.147:4001 , piece_id=RT6JNBA76JSITAX4RQNNMMIQK6LTRRTU3BNO52HGNFFGUXCNTOHA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.642829685 (242h37m20.682402815s) [noise=true , address=81.171.14.70:4045 , piece_id=Z4AIZ4YEZ6MDI3TZG7J7WV2MRNQSYE4DWU2WWH7XIJA3ZFARVNAA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.674811297 (242h37m20.714384417s) [noise=true , address=89.149.201.147:4011 , piece_id=EHMV5WSELKY3MG75UY4PX2LNNYGZF6V666JF6D4BRTTPPDWV2XYA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.679033927 (242h37m20.718607057s) [noise=true , address=89.149.201.147:4019 , piece_id=NF4FWMJXTAANZLOALKFGNBYF3FCAGUA2MUBJ6ODDFG4CKENKN6NA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.682004224 (242h37m20.721577344s) [noise=true , address=81.171.14.70:4041 , piece_id=D72PT76EAJBEBAHC7BARFPW4OIQ7KM3MCWGDC22F56Z6WFG2MEOQ, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.713246368 (242h37m20.752819498s) [noise=true , address=89.149.201.147:4006 , piece_id=BQWUGZR7MOAPRCP7CZO3BN4CXNHOE6JIE4Y37YT25X262HXCBMOQ, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.756094281 (242h37m20.795667411s) [noise=true , address=185.167.204.206:4044 , piece_id=7JX4Z4EDCIQLVHAANDEOCO45UMOWSCFQJWF657GULSY7ENVAYA4Q, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.779235408 (242h37m20.818808538s) [noise=true , address=185.167.204.206:4057 , piece_id=5FEPFMZAMWOTCRDHQ6HOHZ5VXRZIRMLSYCIMYEUFY5D3HIDJXUUQ, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.786701196 (242h37m20.826274316s) [noise=true , address=185.167.204.206:4060 , piece_id=3DD3XYUFZJ4OHPHGI7ZZIGRVXYQM2LN3RIWHT733BCROZYK57NJQ, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.794442146 (242h37m20.834015266s) [noise=true , address=185.167.204.205:4027 , piece_id=G5CYJ2FEPYDMMNRW6ZF5SVEN4WBKIAPTJTLHGIQZAQNAYIPBMGJQ, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.809933235 (242h37m20.849506355s) [noise=true , address=185.167.204.206:4051 , piece_id=PGXE5K6LS3BU72ONIUDPBIWQGLBLCW2ZIRC4RIHAIPB5TQ4BVHXA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.824614171 (242h37m20.864187301s) [noise=true , address=185.167.204.205:4011 , piece_id=L5PD44F7XNZ3IUNL2E7IXI5KS7TMSR654LIBXLEENDYXHNIBYJ2Q, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.829480428 (242h37m20.869053588s) [noise=true , address=185.167.204.206:4038 , piece_id=7LU3S7Q7VNJZOUQDXTUYIK4IN6GSDBPEJN6KOMADYIRF6JVAXFHA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.840763304 (242h37m20.880336424s) [noise=true , address=185.167.204.205:4035 , piece_id=MTDWNDYUQJU6IPMWG223PQFVCSJDSCN52OJZK3Z6URCUSL2RWPZA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.846724636 (242h37m20.886297766s) [noise=true , address=185.167.204.206:4055 , piece_id=ZLTSSUYRLK4X2S52AWMHDDQYW57IWYUNA4DP6LTZXKXWV7QZE3QA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: <nil> Sep 16 16:05:44.846754664 (242h37m20.886327784s) [seg_pos={0 0}, upload=213182]: Segment reached optimal threshold of 26 pieces. Sep 16 16:05:44.846895041 (242h37m20.886468171s) [noise=true , address=185.167.204.205:4031 , piece_id=UPNIRLIUHAJCTRKWHXGHGQFFNPAY3O37BBU7JWNYZXK3FFHOEN4Q, seg_pos={0 0}, upload=213182]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF Sep 16 16:05:44.847098118 (242h37m20.886671238s) [noise=true , address=185.167.204.205:4005 , piece_id=KKKFBGPIG2HU43SQFXCFZAVFXLO5PVG63DPFRMFMITANZCKA7ATA, seg_pos={0 0}, upload=213182]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM): protocol: expected piece hash; context canceled; EOF Sep 16 16:05:44.847140278 (242h37m20.886713398s) [seg_pos={0 0}, upload=213182]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Sep 16 16:05:44.847248714 (242h37m20.886821834s) [upload=213182]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 16 16:05:44.847253363 (242h37m20.886826483s) [upload=213182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 16 16:05:45.102532694 (242h37m21.142105834s) [upload=213182]: Done uploading segments... Sep 16 16:05:45.102567801 (242h37m21.142140931s) [upload=213182]: Upload finished. err: <nil> Duration: 1m0.004256757s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXX.....................................] duration: 6.092814699s FAIL (2025-09-16 16:06:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 36:23.221360752 (242h7m59.260933872s) [seg_pos={0 0}, upload=212758]: Begin upload segment... Sep 16 15:36:23.221427429 (242h7m59.261000559s) [seg_pos={0 0}, upload=212758]: Done begin upload segment. Sep 16 15:36:23.221471934 (242h7m59.261045064s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.221781523 (242h7m59.261354683s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222170084 (242h7m59.261743224s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222380514 (242h7m59.261953644s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222688551 (242h7m59.262261681s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222978833 (242h7m59.262551963s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223291669 (242h7m59.262864809s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223483104 (242h7m59.263056234s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223736246 (242h7m59.263309376s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223952347 (242h7m59.263525477s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224092043 (242h7m59.263665173s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224449704 (242h7m59.264022844s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224537902 (242h7m59.264111032s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224972190 (242h7m59.26454532s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225002056 (242h7m59.264575176s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225500526 (242h7m59.265073646s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225597541 (242h7m59.265170671s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225993644 (242h7m59.265566774s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:24.615369315 (242h8m0.654942445s) [upload=212758]: Next returned nil segment Sep 16 15:36:24.615405684 (242h8m0.654978814s) [upload=212758]: Waiting for error group managing segments... Sep 16 15:36:25.024537642 (242h8m1.064110772s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:07.783420676 (242h8m43.822993807s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:08.809548412 (242h8m44.849121542s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:09.417370950 (242h8m45.45694407s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:10.232853931 (242h8m46.272427062s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.251045936 (242h8m47.290619056s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.530631565 (242h8m47.570204695s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:12.400109127 (242h8m48.439682257s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:13.851060301 (242h8m49.890633431s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.293316228 (242h8m50.332889378s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.648711487 (242h8m50.688284607s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:15.183813908 (242h8m51.223387028s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:19.546944067 (242h8m55.586517197s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:21.938117511 (242h8m57.977690642s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938325488 (242h8m57.977898618s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12EsmL66Gn6K9cUxHqqGtuUZxUGyWCNL4oqyHzS1XUbWVnMMJbd, address:145.79.227.48:4314): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938466356 (242h8m57.978039476s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection Sep 16 15:37:21.938531310 (242h8m57.97810444s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12V9GZR1rr6VfEsK83Bonq2HyceXLcAKnYfoQmA1Qvt5h9u8sik, address:145.79.227.28:4742): protocol: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection Sep 16 15:37:21.938593297 (242h8m57.978166417s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection Sep 16 15:37:21.938613285 (242h8m57.978186405s) [seg_pos={0 0}, upload=212758]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 16 15:37:22.200860966 (242h8m58.240434086s) [upload=212758]: Done uploading segments... Sep 16 15:37:22.200890762 (242h8m58.240463893s) [upload=212758]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.092846409s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 692.053366ms target: https://us1.storj.io/login FAIL (2025-09-16 16:06:40 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 692.084645ms core satellite us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..............................................XX..X..XXX.XXX] duration: 30.00559337s FAIL (2025-09-16 16:06:36 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.005632454s core uplink eu1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 698.132121ms target: https://us1.storj.io/login FAIL (2025-09-16 16:06:24 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 698.205481ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 24 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXX....................................] duration: 6.278712718s FAIL (2025-09-16 16:06:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 36:23.221360752 (242h7m59.260933872s) [seg_pos={0 0}, upload=212758]: Begin upload segment... Sep 16 15:36:23.221427429 (242h7m59.261000559s) [seg_pos={0 0}, upload=212758]: Done begin upload segment. Sep 16 15:36:23.221471934 (242h7m59.261045064s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.221781523 (242h7m59.261354683s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222170084 (242h7m59.261743224s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222380514 (242h7m59.261953644s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222688551 (242h7m59.262261681s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222978833 (242h7m59.262551963s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223291669 (242h7m59.262864809s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223483104 (242h7m59.263056234s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223736246 (242h7m59.263309376s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223952347 (242h7m59.263525477s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224092043 (242h7m59.263665173s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224449704 (242h7m59.264022844s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224537902 (242h7m59.264111032s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224972190 (242h7m59.26454532s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225002056 (242h7m59.264575176s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225500526 (242h7m59.265073646s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225597541 (242h7m59.265170671s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225993644 (242h7m59.265566774s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:24.615369315 (242h8m0.654942445s) [upload=212758]: Next returned nil segment Sep 16 15:36:24.615405684 (242h8m0.654978814s) [upload=212758]: Waiting for error group managing segments... Sep 16 15:36:25.024537642 (242h8m1.064110772s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:07.783420676 (242h8m43.822993807s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:08.809548412 (242h8m44.849121542s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:09.417370950 (242h8m45.45694407s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:10.232853931 (242h8m46.272427062s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.251045936 (242h8m47.290619056s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.530631565 (242h8m47.570204695s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:12.400109127 (242h8m48.439682257s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:13.851060301 (242h8m49.890633431s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.293316228 (242h8m50.332889378s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.648711487 (242h8m50.688284607s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:15.183813908 (242h8m51.223387028s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:19.546944067 (242h8m55.586517197s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:21.938117511 (242h8m57.977690642s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938325488 (242h8m57.977898618s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12EsmL66Gn6K9cUxHqqGtuUZxUGyWCNL4oqyHzS1XUbWVnMMJbd, address:145.79.227.48:4314): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938466356 (242h8m57.978039476s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection Sep 16 15:37:21.938531310 (242h8m57.97810444s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12V9GZR1rr6VfEsK83Bonq2HyceXLcAKnYfoQmA1Qvt5h9u8sik, address:145.79.227.28:4742): protocol: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection Sep 16 15:37:21.938593297 (242h8m57.978166417s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection Sep 16 15:37:21.938613285 (242h8m57.978186405s) [seg_pos={0 0}, upload=212758]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 16 15:37:22.200860966 (242h8m58.240434086s) [upload=212758]: Done uploading segments... Sep 16 15:37:22.200890762 (242h8m58.240463893s) [upload=212758]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.278756252s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...............................................XX..X..XXX.XX] duration: 30.000652613s FAIL (2025-09-16 16:06:00 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000668075s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X....X..XXX.X..XX..........................................] duration: 6.640522413s FAIL (2025-09-16 16:05:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.640549023s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 6 out of 60 is failed (threshold=3) [....................................................XX.X.XXX] duration: 1m0.002872109s FAIL (2025-09-16 16:05:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== DOBHGOUX3AG3YCDC7R6HD4UEHGQXUF26RZC4ZL42RQ, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.668526160 (242h36m18.70809928s) [noise=true , address=185.167.204.205:4024 , piece_id=5U2QO6HSDJPCYUXZARGVVIFN5HXNBEQDSA5TYUZXWVWLQVYGI2PQ, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.669089863 (242h36m18.708662983s) [noise=true , address=185.167.204.206:4051 , piece_id=QUAA6A6JGO3CGN2KPIJFIJEJF7OYVSNVA5XEY2RFQTSWIKBO44GQ, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.669604914 (242h36m18.709178044s) [noise=true , address=81.171.14.70:4026 , piece_id=DJGIN7RIXLNCKXHIFOUSIECN5X7VCLBLNWT2FPDFNJF3AW356CJA, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.670086471 (242h36m18.709659601s) [noise=true , address=89.149.201.147:4002 , piece_id=TBJ7YA7VT7LOBLAV257GD5LCWAHCXSJSME2DH7VTSWTER7HK3SGA, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.670528083 (242h36m18.710101213s) [noise=true , address=185.167.204.205:4031 , piece_id=WTM527XCDZXEEGHZYDJT7OKY3QV4E6KNOVDPLICQSBFEKL4FK3EA, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.671705895 (242h36m18.711279026s) [noise=true , address=89.149.201.147:4010 , piece_id=6VSI4AZSVCWML3F7SOPGSTLHMX3UK6AJWM6TIPFOMCSBXMRM2FIQ, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.672218933 (242h36m18.711792073s) [noise=true , address=89.149.201.147:4005 , piece_id=73IDWYVQLKE3GTP6ALEIMEXD6WPQ6ZJLX2VQUODUQUDYF5CQXFFQ, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.672687384 (242h36m18.712260514s) [noise=true , address=185.167.204.206:4063 , piece_id=PBDDXKGVMMWJ6B3WOBSAREYZ52KBE4UWPN3LMX555WWGU2GIU2NA, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.673420741 (242h36m18.712993871s) [noise=true , address=89.149.201.147:4016 , piece_id=27A4UMXNLEKTZEBRBL27CXMX7Y7ICXZ6MNS3PIK47BUKKD74PBVQ, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.673915954 (242h36m18.713489084s) [noise=true , address=185.167.204.206:4067 , piece_id=7ERFHSWOVLJVWT43A4AI675BNBLS6ERBKHY4KCFMV3QPVXCLF54Q, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.674343568 (242h36m18.713916698s) [noise=true , address=185.167.204.205:4011 , piece_id=SLNIHZLFSHNNF5MQJMXIFDACHN7NGKPV3O7SHHIZRVWOKZNYJN6A, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:42.674843320 (242h36m18.71441644s) [noise=true , address=81.171.14.70:4028 , piece_id=XR7NFTHQERRVEDH4VSIREIYAFFI4BX6Z7XO6ZVDMJR37FCBHUYGA, seg_pos={0 0}, upload=213167]: Uploading piece... Sep 16 16:04:43.526987832 (242h36m19.566560972s) [upload=213167]: Next returned nil segment Sep 16 16:04:43.527021146 (242h36m19.566594276s) [upload=213167]: Waiting for error group managing segments... Sep 16 16:04:43.917437744 (242h36m19.957010874s) [noise=true , address=81.171.14.70:4036 , piece_id=IUN62LELQSL6T56OURWMG744TMJEPLBTOQJFYVMBJYRHNME5XTLA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:43.921641059 (242h36m19.961214179s) [noise=true , address=81.171.14.70:4034 , piece_id=NFWU76ZNIOXSWKVFKCYCFL6OYBNESW4Q4A6ZXFXNKF35ZZ3DIYBA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:43.924296876 (242h36m19.963870006s) [noise=true , address=81.171.14.70:4030 , piece_id=UG3ECEP4JUJBWGHB5U3D5RKP7ZVDU7Q2VGWEGZMFOJB2PPGUR3DA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:43.927568115 (242h36m19.967141245s) [noise=true , address=81.171.14.70:4028 , piece_id=XR7NFTHQERRVEDH4VSIREIYAFFI4BX6Z7XO6ZVDMJR37FCBHUYGA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:43.930515939 (242h36m19.970089069s) [noise=true , address=81.171.14.70:4039 , piece_id=QZMDXWMQ7VTAFPIB2WJ2UIKJQK7F452MZU7CUSVWGAR6UQEEAYHA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:43.955722960 (242h36m19.99529608s) [noise=true , address=81.171.14.70:4040 , piece_id=3VFZLBV73ZMHDBI3SBV4HH3UGKHDPG7HZLTCBLSTWDFF4VIYBVMA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:43.982304097 (242h36m20.021877227s) [noise=true , address=81.171.14.70:4032 , piece_id=FUKEDE6LGCNLDU6242H6OHPWHX62IDQ6J5443DL7YIAOLISXVKDA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:43.997590978 (242h36m20.037164108s) [noise=true , address=89.149.201.147:4005 , piece_id=73IDWYVQLKE3GTP6ALEIMEXD6WPQ6ZJLX2VQUODUQUDYF5CQXFFQ, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.002818171 (242h36m20.042391291s) [noise=true , address=81.171.14.70:4026 , piece_id=DJGIN7RIXLNCKXHIFOUSIECN5X7VCLBLNWT2FPDFNJF3AW356CJA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.009451113 (242h36m20.049024233s) [noise=true , address=89.149.201.147:4008 , piece_id=MQUUAQOR2JW6FMIO3CQ2P6REP2SHRYQSVI5KXQD7AB5OU27JSVQA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.038008895 (242h36m20.077582015s) [noise=true , address=89.149.201.147:4014 , piece_id=EXTUJZ3FE6U3RG5GTWFYQBSFTEQ27INOHS6XRD3WGXHIASN6ORMA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.044400486 (242h36m20.083973616s) [noise=true , address=89.149.201.147:4016 , piece_id=27A4UMXNLEKTZEBRBL27CXMX7Y7ICXZ6MNS3PIK47BUKKD74PBVQ, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.398397962 (242h36m20.437971092s) [noise=true , address=81.171.14.70:4047 , piece_id=GTHOLVAYOQALQJWWXJR3CLUFTX4GYYG4FAWFLCFD3LZEMVAVAN2A, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.422716551 (242h36m20.462289711s) [noise=true , address=89.149.201.147:4010 , piece_id=6VSI4AZSVCWML3F7SOPGSTLHMX3UK6AJWM6TIPFOMCSBXMRM2FIQ, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.447245011 (242h36m20.486818151s) [noise=true , address=89.149.201.147:4013 , piece_id=NKO3CGQX3R66FLNKYGG7Y4QYNZ37GVJDOA5IU6TPUUFTNZGXHICQ, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.472627004 (242h36m20.512200124s) [noise=true , address=185.167.204.205:4031 , piece_id=WTM527XCDZXEEGHZYDJT7OKY3QV4E6KNOVDPLICQSBFEKL4FK3EA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.502048451 (242h36m20.541621581s) [noise=true , address=185.167.204.206:4067 , piece_id=7ERFHSWOVLJVWT43A4AI675BNBLS6ERBKHY4KCFMV3QPVXCLF54Q, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.502395461 (242h36m20.541968591s) [noise=true , address=185.167.204.206:4066 , piece_id=EEWHZMZAKVYHWIQZAV7WSTZEFZU2OFHHTHXOQLGYC32OBKTSD6QQ, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.512969868 (242h36m20.552542998s) [noise=true , address=185.167.204.206:4047 , piece_id=6ZUGK2RNYSCMEI5GAB3MEPGZ7PVPX3NR5NIEVFAXU4ZHFMM2576A, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.513728463 (242h36m20.553301593s) [noise=true , address=89.149.201.147:4002 , piece_id=TBJ7YA7VT7LOBLAV257GD5LCWAHCXSJSME2DH7VTSWTER7HK3SGA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.539272736 (242h36m20.578845856s) [noise=true , address=185.167.204.206:4044 , piece_id=PW3SN6KA2QTQKXDHMF3PTCY7TRXK53U6XKYUZNMHDMV3ZK5XL2QQ, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.542161146 (242h36m20.581734266s) [noise=true , address=185.167.204.206:4063 , piece_id=PBDDXKGVMMWJ6B3WOBSAREYZ52KBE4UWPN3LMX555WWGU2GIU2NA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.568244145 (242h36m20.607817265s) [noise=true , address=185.167.204.205:4024 , piece_id=5U2QO6HSDJPCYUXZARGVVIFN5HXNBEQDSA5TYUZXWVWLQVYGI2PQ, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.576687172 (242h36m20.616260292s) [noise=true , address=185.167.204.206:4051 , piece_id=QUAA6A6JGO3CGN2KPIJFIJEJF7OYVSNVA5XEY2RFQTSWIKBO44GQ, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.596923829 (242h36m20.636496949s) [noise=true , address=185.167.204.205:4027 , piece_id=ZTMKIWEOUMVX6GCVRI6BBMWAMY3Y4N5EAAOU3D6JSSMI6VXMGLPA, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.611278184 (242h36m20.650851314s) [noise=true , address=185.167.204.206:4046 , piece_id=OG6K32IWXJKPSJURXKVH5RV7NFUPMRFRD64BKBPNYOJH4IFHCVQQ, seg_pos={0 0}, upload=213167]: Done uploading piece. err: <nil> Sep 16 16:04:44.611315435 (242h36m20.650888565s) [seg_pos={0 0}, upload=213167]: Segment reached optimal threshold of 26 pieces. Sep 16 16:04:44.611506148 (242h36m20.651079278s) [noise=true , address=185.167.204.205:4011 , piece_id=SLNIHZLFSHNNF5MQJMXIFDACHN7NGKPV3O7SHHIZRVWOKZNYJN6A, seg_pos={0 0}, upload=213167]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3): protocol: expected piece hash; context canceled; EOF Sep 16 16:04:44.611737508 (242h36m20.651310628s) [noise=true , address=185.167.204.206:4045 , piece_id=R5DN6A7CO5DOBHGOUX3AG3YCDC7R6HD4UEHGQXUF26RZC4ZL42RQ, seg_pos={0 0}, upload=213167]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF Sep 16 16:04:44.611791170 (242h36m20.6513643s) [seg_pos={0 0}, upload=213167]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Sep 16 16:04:44.611899787 (242h36m20.651472917s) [upload=213167]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 16 16:04:44.611907172 (242h36m20.651480302s) [upload=213167]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 16 16:04:44.865209708 (242h36m20.904782838s) [upload=213167]: Done uploading segments... Sep 16 16:04:44.865238743 (242h36m20.904811873s) [upload=213167]: Upload finished. err: <nil> Duration: 1m0.002903049s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 7 out of 60 is failed (threshold=3) [................................................XX..X..XXX.X] duration: 30.004840707s FAIL (2025-09-16 16:05:29 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.004867238s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 25 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXX...................................] duration: 6.519581368s FAIL (2025-09-16 16:05:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 36:23.221360752 (242h7m59.260933872s) [seg_pos={0 0}, upload=212758]: Begin upload segment... Sep 16 15:36:23.221427429 (242h7m59.261000559s) [seg_pos={0 0}, upload=212758]: Done begin upload segment. Sep 16 15:36:23.221471934 (242h7m59.261045064s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.221781523 (242h7m59.261354683s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222170084 (242h7m59.261743224s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222380514 (242h7m59.261953644s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222688551 (242h7m59.262261681s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222978833 (242h7m59.262551963s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223291669 (242h7m59.262864809s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223483104 (242h7m59.263056234s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223736246 (242h7m59.263309376s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223952347 (242h7m59.263525477s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224092043 (242h7m59.263665173s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224449704 (242h7m59.264022844s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224537902 (242h7m59.264111032s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224972190 (242h7m59.26454532s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225002056 (242h7m59.264575176s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225500526 (242h7m59.265073646s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225597541 (242h7m59.265170671s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225993644 (242h7m59.265566774s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:24.615369315 (242h8m0.654942445s) [upload=212758]: Next returned nil segment Sep 16 15:36:24.615405684 (242h8m0.654978814s) [upload=212758]: Waiting for error group managing segments... Sep 16 15:36:25.024537642 (242h8m1.064110772s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:07.783420676 (242h8m43.822993807s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:08.809548412 (242h8m44.849121542s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:09.417370950 (242h8m45.45694407s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:10.232853931 (242h8m46.272427062s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.251045936 (242h8m47.290619056s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.530631565 (242h8m47.570204695s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:12.400109127 (242h8m48.439682257s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:13.851060301 (242h8m49.890633431s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.293316228 (242h8m50.332889378s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.648711487 (242h8m50.688284607s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:15.183813908 (242h8m51.223387028s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:19.546944067 (242h8m55.586517197s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:21.938117511 (242h8m57.977690642s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938325488 (242h8m57.977898618s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12EsmL66Gn6K9cUxHqqGtuUZxUGyWCNL4oqyHzS1XUbWVnMMJbd, address:145.79.227.48:4314): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938466356 (242h8m57.978039476s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection Sep 16 15:37:21.938531310 (242h8m57.97810444s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12V9GZR1rr6VfEsK83Bonq2HyceXLcAKnYfoQmA1Qvt5h9u8sik, address:145.79.227.28:4742): protocol: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection Sep 16 15:37:21.938593297 (242h8m57.978166417s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection Sep 16 15:37:21.938613285 (242h8m57.978186405s) [seg_pos={0 0}, upload=212758]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 16 15:37:22.200860966 (242h8m58.240434086s) [upload=212758]: Done uploading segments... Sep 16 15:37:22.200890762 (242h8m58.240463893s) [upload=212758]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.519616975s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 678.30744ms target: https://us1.storj.io/login FAIL (2025-09-16 16:05:12 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 678.333449ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 26 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXX..................................] duration: 6.562627961s FAIL (2025-09-16 16:05:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 36:23.221360752 (242h7m59.260933872s) [seg_pos={0 0}, upload=212758]: Begin upload segment... Sep 16 15:36:23.221427429 (242h7m59.261000559s) [seg_pos={0 0}, upload=212758]: Done begin upload segment. Sep 16 15:36:23.221471934 (242h7m59.261045064s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.221781523 (242h7m59.261354683s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222170084 (242h7m59.261743224s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222380514 (242h7m59.261953644s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222688551 (242h7m59.262261681s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.222978833 (242h7m59.262551963s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223291669 (242h7m59.262864809s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223483104 (242h7m59.263056234s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223736246 (242h7m59.263309376s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.223952347 (242h7m59.263525477s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224092043 (242h7m59.263665173s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224449704 (242h7m59.264022844s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224537902 (242h7m59.264111032s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.224972190 (242h7m59.26454532s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225002056 (242h7m59.264575176s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225500526 (242h7m59.265073646s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225597541 (242h7m59.265170671s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:23.225993644 (242h7m59.265566774s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Uploading piece... Sep 16 15:36:24.615369315 (242h8m0.654942445s) [upload=212758]: Next returned nil segment Sep 16 15:36:24.615405684 (242h8m0.654978814s) [upload=212758]: Waiting for error group managing segments... Sep 16 15:36:25.024537642 (242h8m1.064110772s) [noise=true , address=64.49.13.27:4965 , piece_id=KF4LVZ3S2ZZZS6DGE4JLYJBO654UBLUAMR4W55RIZT4FD532WY3A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:07.783420676 (242h8m43.822993807s) [noise=true , address=145.79.227.25:4636 , piece_id=OK7WSU7W75O4VJKGVJKBTD3I3ZUWAL2RDFCIJOFAPTIKYBUFXUUA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:08.809548412 (242h8m44.849121542s) [noise=true , address=145.79.227.11:4741 , piece_id=6ZA4HBPAKI4I6QKKPX2J7Q6PYJ5XBAPUBOKBNYYOCIWDHF44ABRQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:09.417370950 (242h8m45.45694407s) [noise=true , address=145.79.227.13:4816 , piece_id=KBKSSIPCMITDQ6TZDRV4IUBBLFGXE47ZTMZLZMEJJV6RAD3YQ2PA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:10.232853931 (242h8m46.272427062s) [noise=true , address=145.79.227.36:4546 , piece_id=I4Z5LT4466BGRNEA4LTA3R4RADTLMRXAIVNMR4ZYMBSVQYZ4ZKQA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.251045936 (242h8m47.290619056s) [noise=true , address=145.79.227.49:4369 , piece_id=3KQTFT5TKU6KP244Q3QAHLEBJHBE25XNNVJQBUGXRSXYQWL7X3VA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:11.530631565 (242h8m47.570204695s) [noise=true , address=145.79.227.33:4869 , piece_id=6BBFVXVUSZFFUUOF3OW3YTEEHLRQTTPZSC4ZARW47OOPGC7C4T3Q, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:12.400109127 (242h8m48.439682257s) [noise=true , address=145.79.227.36:4530 , piece_id=OKDVMGQFOX57TJ5BOCUCQHARVVCFXMAWK3ANCFCYXOCBI3JA4ZBA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:13.851060301 (242h8m49.890633431s) [noise=true , address=145.79.227.35:4488 , piece_id=EQQBJMAFW6EA2UEVRNL5JSR6E4TOHLUER3AH6EH5KOVAYR6BTZSQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.293316228 (242h8m50.332889378s) [noise=true , address=145.79.227.52:4468 , piece_id=WRWENTLHQSNMRFJWHE2CF5Y3N6KM55RORNZYEVJWD5KQQGVG6F6A, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:14.648711487 (242h8m50.688284607s) [noise=true , address=145.79.227.12:4769 , piece_id=V2ZYPY4MJRCUAQRHL7XE6OH6SAQLKMPAW2JW5AGI5ZL42MXY7BPA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:15.183813908 (242h8m51.223387028s) [noise=true , address=145.79.227.19:4083 , piece_id=MVRFB3PDI2KTCPPZTUTNXOREB6J4UTD7RG4NNSIYLDTT4Y5ONUCQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:19.546944067 (242h8m55.586517197s) [noise=true , address=145.79.227.30:4792 , piece_id=DQ7IXKMFHDG43D3XWLUMDQLE6SKTPLRRNHHDEYCN7LZ5KRV7MIDA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: <nil> Sep 16 15:37:21.938117511 (242h8m57.977690642s) [noise=true , address=145.79.227.23:4546 , piece_id=3X4C43D6D5R23OZ6VCJAQBOYRJM3DALSHFOVVA3U6OOMLTWJ7DVA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938325488 (242h8m57.977898618s) [noise=true , address=145.79.227.48:4314 , piece_id=7QLHNXRQCOSUMGFMU3NXBPGX7VAHN4SKLOASMC2KNCDVLEXG4ZCA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12EsmL66Gn6K9cUxHqqGtuUZxUGyWCNL4oqyHzS1XUbWVnMMJbd, address:145.79.227.48:4314): protocol: expected piece hash; context deadline exceeded; EOF Sep 16 15:37:21.938466356 (242h8m57.978039476s) [noise=true , address=145.79.227.48:4310 , piece_id=PHF526HVGO2N45JWUVW3GJSM6JEFVETKIGDJJKL2BCB6WQQ2J4HQ, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54210->145.79.227.48:4310: use of closed network connection Sep 16 15:37:21.938531310 (242h8m57.97810444s) [noise=true , address=145.79.227.28:4742 , piece_id=2TPYZPH262XJRPKWGW7BMGUSOCY7ZWEEHVZUPTHJEF47DDSQWZIA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:12V9GZR1rr6VfEsK83Bonq2HyceXLcAKnYfoQmA1Qvt5h9u8sik, address:145.79.227.28:4742): protocol: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46558->145.79.227.28:4742: use of closed network connection Sep 16 15:37:21.938593297 (242h8m57.978166417s) [noise=true , address=145.79.227.2:4131 , piece_id=RSURCLWZHN266HYZ5G2J453BOS3OLO6FI4MRKLE7TVIKZYW3PYOA, seg_pos={0 0}, upload=212758]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51500->145.79.227.2:4131: use of closed network connection Sep 16 15:37:21.938613285 (242h8m57.978186405s) [seg_pos={0 0}, upload=212758]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 16 15:37:22.200860966 (242h8m58.240434086s) [upload=212758]: Done uploading segments... Sep 16 15:37:22.200890762 (242h8m58.240463893s) [upload=212758]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.562661896s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..X....X..XXX.X..XX.........................................] duration: 6.413335517s FAIL (2025-09-16 16:04:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 16 15:36:39.864582971 (242h8m15.904156091s) [upload=212762]: Starting upload Sep 16 15:36:39.864666430 (242h8m15.90423956s) [upload=212762]: Done starting upload Sep 16 15:36:39.864984095 (242h8m15.904557225s) [upload=212762]: Uploading segments... Sep 16 15:36:39.865058366 (242h8m15.904631517s) [upload=212762]: Got next segment. Inline: false Sep 16 15:36:39.865096900 (242h8m15.90467002s) [upload=212762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 15:36:39.865103672 (242h8m15.904676792s) [upload=212762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 15:36:40.131406296 (242h8m16.170979416s) [seg_pos={0 0}, upload=212762]: Begin upload segment... Sep 16 15:36:40.131493051 (242h8m16.171066181s) [seg_pos={0 0}, upload=212762]: Done begin upload segment. Sep 16 15:36:40.131521224 (242h8m16.171094354s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.131965019 (242h8m16.171538149s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132377405 (242h8m16.171950525s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.132720719 (242h8m16.172293839s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133092026 (242h8m16.172665146s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.133508589 (242h8m16.173081719s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134073435 (242h8m16.173646565s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134546686 (242h8m16.174119816s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.134952929 (242h8m16.174526049s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135376456 (242h8m16.174949586s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.135773041 (242h8m16.175346171s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136194754 (242h8m16.175767884s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136652365 (242h8m16.176225485s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.136978938 (242h8m16.176552058s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.137433994 (242h8m16.177007124s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:40.138415523 (242h8m16.177988643s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Uploading piece... Sep 16 15:36:41.918530256 (242h8m17.958103387s) [upload=212762]: Next returned nil segment Sep 16 15:36:41.918552720 (242h8m17.95812585s) [upload=212762]: Waiting for error group managing segments... Sep 16 15:36:42.289194432 (242h8m18.328767552s) [noise=true , address=45.91.116.13:4153 , piece_id=K2DLAKYGGJ7645AK3PS6FCBJK7E5YGKPXLXJPUFK2KB6WQQ5CTUQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.303198951 (242h8m18.342772081s) [noise=true , address=45.91.116.4:4937 , piece_id=6TURWI5DLOOHFHJTXEHBIYKOY7NEYAYQGGFYASRBM4Z7ZADJBWTA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.315161531 (242h8m18.354734661s) [noise=true , address=77.90.31.12:4589 , piece_id=IDTZKXBYH5XIBUPFV7O22BVFIJMXCJ5WU5QXZDC5PPADYV4IARFA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.326500213 (242h8m18.366073343s) [noise=true , address=77.90.31.15:4698 , piece_id=ZKTX4FRUYUW6FBM7R6DW5CO2Z24UBIV5RPBM3CQSEOCNCSEL3VQQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.332147016 (242h8m18.371720136s) [noise=true , address=77.90.31.18:4806 , piece_id=RWKHAKF3PBW4RRZ5FHGIDWK3CVXFQBQSMLI2B6OP4DHIBEAMZRVA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.335516342 (242h8m18.375089472s) [noise=true , address=45.91.116.9:4261 , piece_id=4CV7O3O2P3P6PVFPYXVATAFHHD3MINRIWK7C7DZOGZ7QYTBQDKPQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.339529143 (242h8m18.379102273s) [noise=true , address=45.91.116.7:4010 , piece_id=3THKPSOY6TK456YCJACUIC2NB6ZP5O3LVZNRMGHEY4WMNHNJQDDA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.351282645 (242h8m18.390855775s) [noise=true , address=45.91.116.10:4083 , piece_id=EDLGCDIMAJQNFIGXH72RYVXA553NWSINYJQGTAKPYG5QABRTMD2A, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.376724454 (242h8m18.416297574s) [noise=true , address=45.91.116.14:4190 , piece_id=SCNXLQ53AEJ74FT47E6SLQCQZ73IEJMG437DRVJQUUVN6ND4JWXQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.399079644 (242h8m18.438652774s) [noise=true , address=45.91.116.11:4118 , piece_id=BSQFRJPTRWRAQMIRV4OI56EBGSK5NUBLZOJDCJZQBJ6U7KL7EYSQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:42.524302653 (242h8m18.563875773s) [noise=true , address=45.91.116.13:4154 , piece_id=NK4WEV72WHENUAHUWCB4HSBFYB5WALC3IWCY7LYWERRUIIWCVTXA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:36:43.034623302 (242h8m19.074196432s) [noise=true , address=77.90.31.13:4627 , piece_id=ORRPDXPMEEEMNKX6VF3X453GDF22S4AIKRUTYYKVT5G6E3P2I2QA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: <nil> Sep 16 15:37:09.141416498 (242h8m45.180989638s) [noise=true , address=145.79.227.5:4221 , piece_id=K645GN6ONVUTY7QYA5X4TP5DJXQ5BSXLB3IGYAAB5LWE2C6EWQHA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41968->145.79.227.5:4221: use of closed network connection Sep 16 15:37:09.141469980 (242h8m45.18104311s) [noise=true , address=145.79.227.6:4257 , piece_id=MIT33XKDG3KQU5EVJEF523I4IAWKHAWVXGXZK6KN2UG4OCEKURJQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52094->145.79.227.6:4257: use of closed network connection Sep 16 15:37:09.141486030 (242h8m45.18105915s) [noise=true , address=145.79.227.18:4007 , piece_id=OIHWQLPXV5JY2MGZRZEU3VN5YAOG5TFE5YAPKZ3IH6W22EYIB3YQ, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41180->145.79.227.18:4007: use of closed network connection Sep 16 15:37:09.141522239 (242h8m45.181095369s) [noise=true , address=145.79.227.19:4079 , piece_id=YDKTEFXAFGVFDHZ3VJ7BLAP3VSAGKPIYD7SH2LE3WDSOECANCUIA, seg_pos={0 0}, upload=212762]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59406->145.79.227.19:4079: use of closed network connection Sep 16 15:37:09.141561494 (242h8m45.181134624s) [seg_pos={0 0}, upload=212762]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 16 15:37:09.399175962 (242h8m45.438749102s) [upload=212762]: Done uploading segments... Sep 16 15:37:09.399209846 (242h8m45.438782976s) [upload=212762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.413366466s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [.....................................................XX.X.XX] duration: 1m0.000721701s FAIL (2025-09-16 16:04:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Sep 16 16:03:42.102059458 (242h35m18.141632588s) [upload=213147]: Starting upload Sep 16 16:03:42.102151433 (242h35m18.141724573s) [upload=213147]: Done starting upload Sep 16 16:03:42.102222658 (242h35m18.141795799s) [upload=213147]: Uploading segments... Sep 16 16:03:42.102430725 (242h35m18.142003855s) [upload=213147]: Got next segment. Inline: false Sep 16 16:03:42.102473496 (242h35m18.142046616s) [upload=213147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 16 16:03:42.102480360 (242h35m18.14205349s) [upload=213147]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 16 16:04:41.628254343 (242h36m17.667827473s) [upload=213147]: Next segment err: context deadline exceeded Sep 16 16:04:41.628385672 (242h36m17.667958802s) [upload=213147]: Done uploading segments... Sep 16 16:04:41.628402675 (242h36m17.667975795s) [upload=213147]: Upload finished. err: context deadline exceeded Duration: 1m0.000733714s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)