Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X............X...............] duration: 6.003169325s FAIL (2025-09-06 06:22:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.003191778s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X............X..............] duration: 5.939974125s FAIL (2025-09-06 06:21:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.940003872s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.............X..............] duration: 5.98299362s FAIL (2025-09-06 06:21:54 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.983016315s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X.............X.............] duration: 5.988472934s FAIL (2025-09-06 06:21:18 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.988535404s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X............X.............] duration: 6.381414571s FAIL (2025-09-06 06:21:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.381444388s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.............X............] duration: 6.345861043s FAIL (2025-09-06 06:20:33 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.345883286s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.............X...........] duration: 6.17986561s FAIL (2025-09-06 06:20:17 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.179888955s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X............X............] duration: 6.040414369s FAIL (2025-09-06 06:20:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.040443906s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.............X..........] duration: 6.187956842s FAIL (2025-09-06 06:19:43 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.187983565s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X............X...........] duration: 6.07120225s FAIL (2025-09-06 06:19:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.071234102s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.............X.........] duration: 6.206356034s FAIL (2025-09-06 06:19:23 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.20637976s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X............X..........] duration: 6.173286815s FAIL (2025-09-06 06:19:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.173314967s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.............X........] duration: 6.18424961s FAIL (2025-09-06 06:18:34 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.18427541s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.............X.......] duration: 6.227583205s FAIL (2025-09-06 06:17:58 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.227605287s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X............X.........] duration: 6.195371072s FAIL (2025-09-06 06:17:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.195394707s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.............X......] duration: 6.190854086s FAIL (2025-09-06 06:17:14 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.190907588s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X............X........] duration: 6.042614569s FAIL (2025-09-06 06:17:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.042644057s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.............X.....] duration: 6.000988722s FAIL (2025-09-06 06:16:00 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.001005355s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X............X.......] duration: 6.152204129s FAIL (2025-09-06 06:15:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.152229278s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X.............X....] duration: 5.798409031s FAIL (2025-09-06 06:15:18 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.798454609s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X............X......] duration: 6.312910276s FAIL (2025-09-06 06:15:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.312939471s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X.............X...] duration: 6.369529513s FAIL (2025-09-06 06:14:36 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.369553159s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X............X.....] duration: 6.380388964s FAIL (2025-09-06 06:14:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.380416016s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X.............X..] duration: 5.965738426s FAIL (2025-09-06 06:13:55 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.965763434s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X............X....] duration: 6.238103373s FAIL (2025-09-06 06:13:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.238128553s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X............X...] duration: 6.4469759s FAIL (2025-09-06 06:13:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.447003372s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X............X..] duration: 6.353590252s FAIL (2025-09-06 06:13:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.353613876s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X.............X.] duration: 6.140751091s FAIL (2025-09-06 06:13:12 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.140774155s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X............X.] duration: 7.770811811s FAIL (2025-09-06 06:12:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.770862258s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X............X] duration: 31.093279858s FAIL (2025-09-06 06:12:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 06:12:01.700822077 (184h3m58.381017646s) [upload=161970]: Starting upload Sep 6 06:12:01.700918110 (184h3m58.381113689s) [upload=161970]: Done starting upload Sep 6 06:12:01.700999997 (184h3m58.381195586s) [upload=161970]: Uploading segments... Sep 6 06:12:01.701266497 (184h3m58.381462076s) [upload=161970]: Got next segment. Inline: false Sep 6 06:12:01.701308627 (184h3m58.381504206s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:12:01.701314359 (184h3m58.381509928s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:12:02.475732664 (184h3m59.155928243s) [seg_pos={0 0}, upload=161970]: Begin upload segment... Sep 6 06:12:02.475790845 (184h3m59.155986434s) [seg_pos={0 0}, upload=161970]: Done begin upload segment. Sep 6 06:12:02.475820431 (184h3m59.156016011s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.475952194 (184h3m59.156147763s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476305560 (184h3m59.156501139s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476314186 (184h3m59.156509755s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476575627 (184h3m59.156771206s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476731746 (184h3m59.156927325s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.476853478 (184h3m59.157049047s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477137842 (184h3m59.157333411s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477176316 (184h3m59.157371895s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477390597 (184h3m59.157586166s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477573556 (184h3m59.157769135s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.477981547 (184h3m59.158177116s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478337768 (184h3m59.158533347s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.478685925 (184h3m59.158881504s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479080280 (184h3m59.159275859s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:02.479905709 (184h3m59.160101278s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Uploading piece... Sep 6 06:12:06.743681894 (184h4m3.423877463s) [upload=161970]: Next returned nil segment Sep 6 06:12:06.743698256 (184h4m3.423893825s) [upload=161970]: Waiting for error group managing segments... Sep 6 06:12:08.566147452 (184h4m5.246343021s) [noise=true , address=45.91.116.6:4974 , piece_id=26RG5U442BRIATJEFGUXD4S67CUM7RYEDF4HMN2373IMCVL2SZ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:10.525093423 (184h4m7.205288992s) [noise=true , address=77.90.31.21:4879 , piece_id=LQ6KSUGIYNT63JXHS4GVOZ5AEDOQBDWBKVMPD3KZQ4R3GRYDBAFQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.354397637 (184h4m8.034593206s) [noise=true , address=45.91.116.12:4225 , piece_id=CXSKY4BHNMIHRHENJVC77DP7J76TREQDJZVXNUI3O3QDMIKNCVYA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.395272754 (184h4m8.075468323s) [noise=true , address=45.91.116.5:4299 , piece_id=EMRHS7F27YVEPYP6JBN64TQBE4BA4DKODZXXW5LJ5LY4O2VQQKJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:11.790455180 (184h4m8.470650749s) [noise=true , address=77.90.31.14:4661 , piece_id=OFAMWXPET2VMBUWMP7GUPO5AXP6FPX4TQEDBBTTG34K4XSXO55HQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.483284611 (184h4m9.16348018s) [noise=true , address=45.91.116.4:4937 , piece_id=KBHPO54GRW266FNC6CQOUZFJH5O573FNBUHEBNQ3MIXNNHYFHRVA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:12.869691793 (184h4m9.549887362s) [noise=true , address=45.91.116.5:4297 , piece_id=RNUJY7KQ7E7BJZCFMH7F673SBU6QEBMQJIQ4UQCR2Z6QCOAWITLA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.318260547 (184h4m9.998456116s) [noise=true , address=45.91.116.6:4975 , piece_id=BI366PPQAATDBOXCJKP5MTJ6KKDNU2BFR4KURGO5RXOWVUX73JQQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:13.850428602 (184h4m10.530624161s) [noise=true , address=45.91.116.9:4263 , piece_id=UNPH3NAP3G7KPFAOKURXNFDQCIGJ2E5UF3SOZXWUV7322V5DAEDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.393334224 (184h4m11.073529793s) [noise=true , address=45.91.116.13:4153 , piece_id=MWZWO5IXDYXYUDWQB6VA5JHJJ5GOBE5DEBOALEUSIOVYMITEUQ4Q, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:14.425538008 (184h4m11.105733567s) [noise=true , address=45.91.116.9:4261 , piece_id=BP7B2C4YN27YLBT4H3C5LJPP5BAB4D7DSBVWN5Z6XAD2QFBUIAJQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:21.744423683 (184h4m18.424619252s) [noise=true , address=77.90.31.15:4698 , piece_id=YKGIN25VWSAOW35N3GBKHQOS6FV7XYREG4TKSJZFAS7D6MIRWGDQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068215417 (184h4m24.748410986s) [noise=true , address=77.90.31.15:4697 , piece_id=Q3FI363PXMYHKPSILKY37G3S2YT2LQAZLBWKM6HACP26DRYZTYFA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: <nil> Sep 6 06:12:28.068247438 (184h4m24.748443017s) [seg_pos={0 0}, upload=161970]: Segment reached optimal threshold of 13 pieces. Sep 6 06:12:28.068433614 (184h4m24.748629183s) [noise=true , address=45.91.116.10:4081 , piece_id=SXJZWLMRXCACP7DACOIEYKHQIBTHEWX5T37WQLYVYB6GMVHRWFIA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068460706 (184h4m24.748656275s) [noise=true , address=45.91.116.12:4226 , piece_id=TU5QHN6WJ2CTRI3SDN3CSHP6W7TBGWO7GI52DTUGNQ3S35F2ZOGQ, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 06:12:28.068513476 (184h4m24.748709045s) [noise=true , address=45.91.116.13:4155 , piece_id=TTECFLVVB3WTEXL766K22F6PXGFOXUPXTBU7CQNS35D2GPGQRHJA, seg_pos={0 0}, upload=161970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58250->45.91.116.13:4155: use of closed network connection Sep 6 06:12:28.068530459 (184h4m24.748726028s) [seg_pos={0 0}, upload=161970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 06:12:28.068624640 (184h4m24.748820209s) [upload=161970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 06:12:28.068629268 (184h4m24.748824837s) [upload=161970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 06:12:30.180207168 (184h4m26.860402737s) [upload=161970]: Done uploading segments... Sep 6 06:12:30.180234631 (184h4m26.8604302s) [upload=161970]: Upload finished. err: metaclient: context deadline exceeded Duration: 31.093299286s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.............X] duration: 30.001353152s FAIL (2025-09-06 06:12:25 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001369031s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.364735981s FAIL (2025-09-06 06:11:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.364752241s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.227875699s FAIL (2025-09-06 06:11:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 6 06:00:57.409324367 (183h52m54.089519936s) [upload=161812]: Starting upload Sep 6 06:00:57.409382077 (183h52m54.089577636s) [upload=161812]: Done starting upload Sep 6 06:00:57.409445548 (183h52m54.089641117s) [upload=161812]: Uploading segments... Sep 6 06:00:57.409559547 (183h52m54.089755116s) [upload=161812]: Got next segment. Inline: false Sep 6 06:00:57.409585677 (183h52m54.089781246s) [upload=161812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:00:57.409590186 (183h52m54.089785755s) [upload=161812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:00:57.944556245 (183h52m54.624751814s) [seg_pos={0 0}, upload=161812]: Begin upload segment... Sep 6 06:00:57.944600689 (183h52m54.624796258s) [seg_pos={0 0}, upload=161812]: Done begin upload segment. Sep 6 06:00:57.944622582 (183h52m54.624818151s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.944696784 (183h52m54.624892353s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945040401 (183h52m54.62523597s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945121205 (183h52m54.625316774s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945313844 (183h52m54.625509413s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945385632 (183h52m54.625581201s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945564755 (183h52m54.625760324s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945640088 (183h52m54.625835657s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945877623 (183h52m54.626073192s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945932328 (183h52m54.626127897s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946134234 (183h52m54.626329803s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946178168 (183h52m54.626373737s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946371358 (183h52m54.626566927s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946536604 (183h52m54.626732173s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946675369 (183h52m54.626870949s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.947039156 (183h52m54.627234725s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:01:03.980996351 (183h53m0.66119192s) [upload=161812]: Next returned nil segment Sep 6 06:01:03.981015396 (183h53m0.661210965s) [upload=161812]: Waiting for error group managing segments... Sep 6 06:01:05.190801870 (183h53m1.870997439s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.353458414 (183h53m3.033653983s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.618637531 (183h53m3.2988331s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.074977606 (183h53m4.755173175s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.193455912 (183h53m4.873651481s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.606231531 (183h53m6.2864271s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.729210390 (183h53m6.409405959s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:12.441592321 (183h53m9.12178789s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:17.884919977 (183h53m14.565115547s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.848711975 (183h53m16.528907544s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.979812743 (183h53m16.660008312s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:22.837087414 (183h53m19.517282983s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:24.061391504 (183h53m20.741587073s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection Sep 6 06:01:24.061448052 (183h53m20.741643621s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection Sep 6 06:01:24.061491345 (183h53m20.741686914s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection Sep 6 06:01:24.061521442 (183h53m20.741717011s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection Sep 6 06:01:24.061539307 (183h53m20.741734876s) [seg_pos={0 0}, upload=161812]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 6 06:01:25.813674082 (183h53m22.493869652s) [upload=161812]: Done uploading segments... Sep 6 06:01:25.813704481 (183h53m22.49390007s) [upload=161812]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.227909595s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.39096829s FAIL (2025-09-06 06:10:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.390995282s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.1584458s FAIL (2025-09-06 06:10:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 6 06:00:57.409324367 (183h52m54.089519936s) [upload=161812]: Starting upload Sep 6 06:00:57.409382077 (183h52m54.089577636s) [upload=161812]: Done starting upload Sep 6 06:00:57.409445548 (183h52m54.089641117s) [upload=161812]: Uploading segments... Sep 6 06:00:57.409559547 (183h52m54.089755116s) [upload=161812]: Got next segment. Inline: false Sep 6 06:00:57.409585677 (183h52m54.089781246s) [upload=161812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:00:57.409590186 (183h52m54.089785755s) [upload=161812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:00:57.944556245 (183h52m54.624751814s) [seg_pos={0 0}, upload=161812]: Begin upload segment... Sep 6 06:00:57.944600689 (183h52m54.624796258s) [seg_pos={0 0}, upload=161812]: Done begin upload segment. Sep 6 06:00:57.944622582 (183h52m54.624818151s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.944696784 (183h52m54.624892353s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945040401 (183h52m54.62523597s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945121205 (183h52m54.625316774s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945313844 (183h52m54.625509413s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945385632 (183h52m54.625581201s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945564755 (183h52m54.625760324s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945640088 (183h52m54.625835657s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945877623 (183h52m54.626073192s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945932328 (183h52m54.626127897s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946134234 (183h52m54.626329803s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946178168 (183h52m54.626373737s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946371358 (183h52m54.626566927s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946536604 (183h52m54.626732173s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946675369 (183h52m54.626870949s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.947039156 (183h52m54.627234725s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:01:03.980996351 (183h53m0.66119192s) [upload=161812]: Next returned nil segment Sep 6 06:01:03.981015396 (183h53m0.661210965s) [upload=161812]: Waiting for error group managing segments... Sep 6 06:01:05.190801870 (183h53m1.870997439s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.353458414 (183h53m3.033653983s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.618637531 (183h53m3.2988331s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.074977606 (183h53m4.755173175s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.193455912 (183h53m4.873651481s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.606231531 (183h53m6.2864271s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.729210390 (183h53m6.409405959s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:12.441592321 (183h53m9.12178789s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:17.884919977 (183h53m14.565115547s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.848711975 (183h53m16.528907544s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.979812743 (183h53m16.660008312s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:22.837087414 (183h53m19.517282983s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:24.061391504 (183h53m20.741587073s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection Sep 6 06:01:24.061448052 (183h53m20.741643621s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection Sep 6 06:01:24.061491345 (183h53m20.741686914s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection Sep 6 06:01:24.061521442 (183h53m20.741717011s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection Sep 6 06:01:24.061539307 (183h53m20.741734876s) [seg_pos={0 0}, upload=161812]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 6 06:01:25.813674082 (183h53m22.493869652s) [upload=161812]: Done uploading segments... Sep 6 06:01:25.813704481 (183h53m22.49390007s) [upload=161812]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.158472422s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.21459077s FAIL (2025-09-06 06:09:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.214613292s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.847136764s FAIL (2025-09-06 06:09:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.847156351s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.238631003s FAIL (2025-09-06 06:09:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.238656663s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.368746827s FAIL (2025-09-06 06:09:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 6 06:00:57.409324367 (183h52m54.089519936s) [upload=161812]: Starting upload Sep 6 06:00:57.409382077 (183h52m54.089577636s) [upload=161812]: Done starting upload Sep 6 06:00:57.409445548 (183h52m54.089641117s) [upload=161812]: Uploading segments... Sep 6 06:00:57.409559547 (183h52m54.089755116s) [upload=161812]: Got next segment. Inline: false Sep 6 06:00:57.409585677 (183h52m54.089781246s) [upload=161812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:00:57.409590186 (183h52m54.089785755s) [upload=161812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:00:57.944556245 (183h52m54.624751814s) [seg_pos={0 0}, upload=161812]: Begin upload segment... Sep 6 06:00:57.944600689 (183h52m54.624796258s) [seg_pos={0 0}, upload=161812]: Done begin upload segment. Sep 6 06:00:57.944622582 (183h52m54.624818151s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.944696784 (183h52m54.624892353s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945040401 (183h52m54.62523597s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945121205 (183h52m54.625316774s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945313844 (183h52m54.625509413s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945385632 (183h52m54.625581201s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945564755 (183h52m54.625760324s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945640088 (183h52m54.625835657s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945877623 (183h52m54.626073192s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945932328 (183h52m54.626127897s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946134234 (183h52m54.626329803s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946178168 (183h52m54.626373737s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946371358 (183h52m54.626566927s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946536604 (183h52m54.626732173s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946675369 (183h52m54.626870949s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.947039156 (183h52m54.627234725s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:01:03.980996351 (183h53m0.66119192s) [upload=161812]: Next returned nil segment Sep 6 06:01:03.981015396 (183h53m0.661210965s) [upload=161812]: Waiting for error group managing segments... Sep 6 06:01:05.190801870 (183h53m1.870997439s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.353458414 (183h53m3.033653983s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.618637531 (183h53m3.2988331s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.074977606 (183h53m4.755173175s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.193455912 (183h53m4.873651481s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.606231531 (183h53m6.2864271s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.729210390 (183h53m6.409405959s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:12.441592321 (183h53m9.12178789s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:17.884919977 (183h53m14.565115547s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.848711975 (183h53m16.528907544s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.979812743 (183h53m16.660008312s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:22.837087414 (183h53m19.517282983s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:24.061391504 (183h53m20.741587073s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection Sep 6 06:01:24.061448052 (183h53m20.741643621s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection Sep 6 06:01:24.061491345 (183h53m20.741686914s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection Sep 6 06:01:24.061521442 (183h53m20.741717011s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection Sep 6 06:01:24.061539307 (183h53m20.741734876s) [seg_pos={0 0}, upload=161812]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 6 06:01:25.813674082 (183h53m22.493869652s) [upload=161812]: Done uploading segments... Sep 6 06:01:25.813704481 (183h53m22.49390007s) [upload=161812]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.368773447s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.166008649s FAIL (2025-09-06 06:08:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 6 06:00:57.409324367 (183h52m54.089519936s) [upload=161812]: Starting upload Sep 6 06:00:57.409382077 (183h52m54.089577636s) [upload=161812]: Done starting upload Sep 6 06:00:57.409445548 (183h52m54.089641117s) [upload=161812]: Uploading segments... Sep 6 06:00:57.409559547 (183h52m54.089755116s) [upload=161812]: Got next segment. Inline: false Sep 6 06:00:57.409585677 (183h52m54.089781246s) [upload=161812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:00:57.409590186 (183h52m54.089785755s) [upload=161812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:00:57.944556245 (183h52m54.624751814s) [seg_pos={0 0}, upload=161812]: Begin upload segment... Sep 6 06:00:57.944600689 (183h52m54.624796258s) [seg_pos={0 0}, upload=161812]: Done begin upload segment. Sep 6 06:00:57.944622582 (183h52m54.624818151s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.944696784 (183h52m54.624892353s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945040401 (183h52m54.62523597s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945121205 (183h52m54.625316774s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945313844 (183h52m54.625509413s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945385632 (183h52m54.625581201s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945564755 (183h52m54.625760324s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945640088 (183h52m54.625835657s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945877623 (183h52m54.626073192s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945932328 (183h52m54.626127897s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946134234 (183h52m54.626329803s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946178168 (183h52m54.626373737s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946371358 (183h52m54.626566927s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946536604 (183h52m54.626732173s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946675369 (183h52m54.626870949s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.947039156 (183h52m54.627234725s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:01:03.980996351 (183h53m0.66119192s) [upload=161812]: Next returned nil segment Sep 6 06:01:03.981015396 (183h53m0.661210965s) [upload=161812]: Waiting for error group managing segments... Sep 6 06:01:05.190801870 (183h53m1.870997439s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.353458414 (183h53m3.033653983s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.618637531 (183h53m3.2988331s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.074977606 (183h53m4.755173175s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.193455912 (183h53m4.873651481s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.606231531 (183h53m6.2864271s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.729210390 (183h53m6.409405959s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:12.441592321 (183h53m9.12178789s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:17.884919977 (183h53m14.565115547s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.848711975 (183h53m16.528907544s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.979812743 (183h53m16.660008312s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:22.837087414 (183h53m19.517282983s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:24.061391504 (183h53m20.741587073s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection Sep 6 06:01:24.061448052 (183h53m20.741643621s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection Sep 6 06:01:24.061491345 (183h53m20.741686914s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection Sep 6 06:01:24.061521442 (183h53m20.741717011s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection Sep 6 06:01:24.061539307 (183h53m20.741734876s) [seg_pos={0 0}, upload=161812]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 6 06:01:25.813674082 (183h53m22.493869652s) [upload=161812]: Done uploading segments... Sep 6 06:01:25.813704481 (183h53m22.49390007s) [upload=161812]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.166048475s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.387451786s FAIL (2025-09-06 06:07:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.38746931s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.218373756s FAIL (2025-09-06 06:07:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 6 06:00:57.409324367 (183h52m54.089519936s) [upload=161812]: Starting upload Sep 6 06:00:57.409382077 (183h52m54.089577636s) [upload=161812]: Done starting upload Sep 6 06:00:57.409445548 (183h52m54.089641117s) [upload=161812]: Uploading segments... Sep 6 06:00:57.409559547 (183h52m54.089755116s) [upload=161812]: Got next segment. Inline: false Sep 6 06:00:57.409585677 (183h52m54.089781246s) [upload=161812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:00:57.409590186 (183h52m54.089785755s) [upload=161812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:00:57.944556245 (183h52m54.624751814s) [seg_pos={0 0}, upload=161812]: Begin upload segment... Sep 6 06:00:57.944600689 (183h52m54.624796258s) [seg_pos={0 0}, upload=161812]: Done begin upload segment. Sep 6 06:00:57.944622582 (183h52m54.624818151s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.944696784 (183h52m54.624892353s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945040401 (183h52m54.62523597s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945121205 (183h52m54.625316774s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945313844 (183h52m54.625509413s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945385632 (183h52m54.625581201s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945564755 (183h52m54.625760324s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945640088 (183h52m54.625835657s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945877623 (183h52m54.626073192s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945932328 (183h52m54.626127897s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946134234 (183h52m54.626329803s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946178168 (183h52m54.626373737s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946371358 (183h52m54.626566927s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946536604 (183h52m54.626732173s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946675369 (183h52m54.626870949s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.947039156 (183h52m54.627234725s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:01:03.980996351 (183h53m0.66119192s) [upload=161812]: Next returned nil segment Sep 6 06:01:03.981015396 (183h53m0.661210965s) [upload=161812]: Waiting for error group managing segments... Sep 6 06:01:05.190801870 (183h53m1.870997439s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.353458414 (183h53m3.033653983s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.618637531 (183h53m3.2988331s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.074977606 (183h53m4.755173175s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.193455912 (183h53m4.873651481s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.606231531 (183h53m6.2864271s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.729210390 (183h53m6.409405959s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:12.441592321 (183h53m9.12178789s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:17.884919977 (183h53m14.565115547s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.848711975 (183h53m16.528907544s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.979812743 (183h53m16.660008312s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:22.837087414 (183h53m19.517282983s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:24.061391504 (183h53m20.741587073s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection Sep 6 06:01:24.061448052 (183h53m20.741643621s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection Sep 6 06:01:24.061491345 (183h53m20.741686914s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection Sep 6 06:01:24.061521442 (183h53m20.741717011s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection Sep 6 06:01:24.061539307 (183h53m20.741734876s) [seg_pos={0 0}, upload=161812]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 6 06:01:25.813674082 (183h53m22.493869652s) [upload=161812]: Done uploading segments... Sep 6 06:01:25.813704481 (183h53m22.49390007s) [upload=161812]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.218398183s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.262933026s FAIL (2025-09-06 06:07:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.262962862s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.404569683s FAIL (2025-09-06 06:06:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 6 06:00:57.409324367 (183h52m54.089519936s) [upload=161812]: Starting upload Sep 6 06:00:57.409382077 (183h52m54.089577636s) [upload=161812]: Done starting upload Sep 6 06:00:57.409445548 (183h52m54.089641117s) [upload=161812]: Uploading segments... Sep 6 06:00:57.409559547 (183h52m54.089755116s) [upload=161812]: Got next segment. Inline: false Sep 6 06:00:57.409585677 (183h52m54.089781246s) [upload=161812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:00:57.409590186 (183h52m54.089785755s) [upload=161812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:00:57.944556245 (183h52m54.624751814s) [seg_pos={0 0}, upload=161812]: Begin upload segment... Sep 6 06:00:57.944600689 (183h52m54.624796258s) [seg_pos={0 0}, upload=161812]: Done begin upload segment. Sep 6 06:00:57.944622582 (183h52m54.624818151s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.944696784 (183h52m54.624892353s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945040401 (183h52m54.62523597s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945121205 (183h52m54.625316774s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945313844 (183h52m54.625509413s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945385632 (183h52m54.625581201s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945564755 (183h52m54.625760324s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945640088 (183h52m54.625835657s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945877623 (183h52m54.626073192s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945932328 (183h52m54.626127897s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946134234 (183h52m54.626329803s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946178168 (183h52m54.626373737s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946371358 (183h52m54.626566927s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946536604 (183h52m54.626732173s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946675369 (183h52m54.626870949s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.947039156 (183h52m54.627234725s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:01:03.980996351 (183h53m0.66119192s) [upload=161812]: Next returned nil segment Sep 6 06:01:03.981015396 (183h53m0.661210965s) [upload=161812]: Waiting for error group managing segments... Sep 6 06:01:05.190801870 (183h53m1.870997439s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.353458414 (183h53m3.033653983s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.618637531 (183h53m3.2988331s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.074977606 (183h53m4.755173175s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.193455912 (183h53m4.873651481s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.606231531 (183h53m6.2864271s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.729210390 (183h53m6.409405959s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:12.441592321 (183h53m9.12178789s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:17.884919977 (183h53m14.565115547s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.848711975 (183h53m16.528907544s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.979812743 (183h53m16.660008312s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:22.837087414 (183h53m19.517282983s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:24.061391504 (183h53m20.741587073s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection Sep 6 06:01:24.061448052 (183h53m20.741643621s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection Sep 6 06:01:24.061491345 (183h53m20.741686914s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection Sep 6 06:01:24.061521442 (183h53m20.741717011s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection Sep 6 06:01:24.061539307 (183h53m20.741734876s) [seg_pos={0 0}, upload=161812]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 6 06:01:25.813674082 (183h53m22.493869652s) [upload=161812]: Done uploading segments... Sep 6 06:01:25.813704481 (183h53m22.49390007s) [upload=161812]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.404591015s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.295678588s FAIL (2025-09-06 06:06:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.295704908s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.001628262s FAIL (2025-09-06 06:06:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 6 06:00:57.409324367 (183h52m54.089519936s) [upload=161812]: Starting upload Sep 6 06:00:57.409382077 (183h52m54.089577636s) [upload=161812]: Done starting upload Sep 6 06:00:57.409445548 (183h52m54.089641117s) [upload=161812]: Uploading segments... Sep 6 06:00:57.409559547 (183h52m54.089755116s) [upload=161812]: Got next segment. Inline: false Sep 6 06:00:57.409585677 (183h52m54.089781246s) [upload=161812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:00:57.409590186 (183h52m54.089785755s) [upload=161812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:00:57.944556245 (183h52m54.624751814s) [seg_pos={0 0}, upload=161812]: Begin upload segment... Sep 6 06:00:57.944600689 (183h52m54.624796258s) [seg_pos={0 0}, upload=161812]: Done begin upload segment. Sep 6 06:00:57.944622582 (183h52m54.624818151s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.944696784 (183h52m54.624892353s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945040401 (183h52m54.62523597s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945121205 (183h52m54.625316774s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945313844 (183h52m54.625509413s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945385632 (183h52m54.625581201s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945564755 (183h52m54.625760324s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945640088 (183h52m54.625835657s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945877623 (183h52m54.626073192s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945932328 (183h52m54.626127897s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946134234 (183h52m54.626329803s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946178168 (183h52m54.626373737s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946371358 (183h52m54.626566927s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946536604 (183h52m54.626732173s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946675369 (183h52m54.626870949s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.947039156 (183h52m54.627234725s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:01:03.980996351 (183h53m0.66119192s) [upload=161812]: Next returned nil segment Sep 6 06:01:03.981015396 (183h53m0.661210965s) [upload=161812]: Waiting for error group managing segments... Sep 6 06:01:05.190801870 (183h53m1.870997439s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.353458414 (183h53m3.033653983s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.618637531 (183h53m3.2988331s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.074977606 (183h53m4.755173175s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.193455912 (183h53m4.873651481s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.606231531 (183h53m6.2864271s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.729210390 (183h53m6.409405959s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:12.441592321 (183h53m9.12178789s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:17.884919977 (183h53m14.565115547s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.848711975 (183h53m16.528907544s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.979812743 (183h53m16.660008312s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:22.837087414 (183h53m19.517282983s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:24.061391504 (183h53m20.741587073s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection Sep 6 06:01:24.061448052 (183h53m20.741643621s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection Sep 6 06:01:24.061491345 (183h53m20.741686914s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection Sep 6 06:01:24.061521442 (183h53m20.741717011s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection Sep 6 06:01:24.061539307 (183h53m20.741734876s) [seg_pos={0 0}, upload=161812]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 6 06:01:25.813674082 (183h53m22.493869652s) [upload=161812]: Done uploading segments... Sep 6 06:01:25.813704481 (183h53m22.49390007s) [upload=161812]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.001659451s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.403053384s FAIL (2025-09-06 06:05:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.40307701s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.231278091s FAIL (2025-09-06 06:05:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 6 06:00:57.409324367 (183h52m54.089519936s) [upload=161812]: Starting upload Sep 6 06:00:57.409382077 (183h52m54.089577636s) [upload=161812]: Done starting upload Sep 6 06:00:57.409445548 (183h52m54.089641117s) [upload=161812]: Uploading segments... Sep 6 06:00:57.409559547 (183h52m54.089755116s) [upload=161812]: Got next segment. Inline: false Sep 6 06:00:57.409585677 (183h52m54.089781246s) [upload=161812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:00:57.409590186 (183h52m54.089785755s) [upload=161812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:00:57.944556245 (183h52m54.624751814s) [seg_pos={0 0}, upload=161812]: Begin upload segment... Sep 6 06:00:57.944600689 (183h52m54.624796258s) [seg_pos={0 0}, upload=161812]: Done begin upload segment. Sep 6 06:00:57.944622582 (183h52m54.624818151s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.944696784 (183h52m54.624892353s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945040401 (183h52m54.62523597s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945121205 (183h52m54.625316774s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945313844 (183h52m54.625509413s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945385632 (183h52m54.625581201s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945564755 (183h52m54.625760324s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945640088 (183h52m54.625835657s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945877623 (183h52m54.626073192s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945932328 (183h52m54.626127897s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946134234 (183h52m54.626329803s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946178168 (183h52m54.626373737s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946371358 (183h52m54.626566927s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946536604 (183h52m54.626732173s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946675369 (183h52m54.626870949s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.947039156 (183h52m54.627234725s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:01:03.980996351 (183h53m0.66119192s) [upload=161812]: Next returned nil segment Sep 6 06:01:03.981015396 (183h53m0.661210965s) [upload=161812]: Waiting for error group managing segments... Sep 6 06:01:05.190801870 (183h53m1.870997439s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.353458414 (183h53m3.033653983s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.618637531 (183h53m3.2988331s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.074977606 (183h53m4.755173175s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.193455912 (183h53m4.873651481s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.606231531 (183h53m6.2864271s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.729210390 (183h53m6.409405959s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:12.441592321 (183h53m9.12178789s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:17.884919977 (183h53m14.565115547s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.848711975 (183h53m16.528907544s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.979812743 (183h53m16.660008312s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:22.837087414 (183h53m19.517282983s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:24.061391504 (183h53m20.741587073s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection Sep 6 06:01:24.061448052 (183h53m20.741643621s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection Sep 6 06:01:24.061491345 (183h53m20.741686914s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection Sep 6 06:01:24.061521442 (183h53m20.741717011s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection Sep 6 06:01:24.061539307 (183h53m20.741734876s) [seg_pos={0 0}, upload=161812]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 6 06:01:25.813674082 (183h53m22.493869652s) [upload=161812]: Done uploading segments... Sep 6 06:01:25.813704481 (183h53m22.49390007s) [upload=161812]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.231297128s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.427391881s FAIL (2025-09-06 06:05:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 6 06:00:57.409324367 (183h52m54.089519936s) [upload=161812]: Starting upload Sep 6 06:00:57.409382077 (183h52m54.089577636s) [upload=161812]: Done starting upload Sep 6 06:00:57.409445548 (183h52m54.089641117s) [upload=161812]: Uploading segments... Sep 6 06:00:57.409559547 (183h52m54.089755116s) [upload=161812]: Got next segment. Inline: false Sep 6 06:00:57.409585677 (183h52m54.089781246s) [upload=161812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 06:00:57.409590186 (183h52m54.089785755s) [upload=161812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 06:00:57.944556245 (183h52m54.624751814s) [seg_pos={0 0}, upload=161812]: Begin upload segment... Sep 6 06:00:57.944600689 (183h52m54.624796258s) [seg_pos={0 0}, upload=161812]: Done begin upload segment. Sep 6 06:00:57.944622582 (183h52m54.624818151s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.944696784 (183h52m54.624892353s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945040401 (183h52m54.62523597s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945121205 (183h52m54.625316774s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945313844 (183h52m54.625509413s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945385632 (183h52m54.625581201s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945564755 (183h52m54.625760324s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945640088 (183h52m54.625835657s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945877623 (183h52m54.626073192s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.945932328 (183h52m54.626127897s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946134234 (183h52m54.626329803s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946178168 (183h52m54.626373737s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946371358 (183h52m54.626566927s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946536604 (183h52m54.626732173s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.946675369 (183h52m54.626870949s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:00:57.947039156 (183h52m54.627234725s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Uploading piece... Sep 6 06:01:03.980996351 (183h53m0.66119192s) [upload=161812]: Next returned nil segment Sep 6 06:01:03.981015396 (183h53m0.661210965s) [upload=161812]: Waiting for error group managing segments... Sep 6 06:01:05.190801870 (183h53m1.870997439s) [noise=true , address=77.90.31.15:4699 , piece_id=5Y33UMG22SD3GHJ6Z4JBK7YIA4N64GDXUOLUBUP2RAMKWOGOHDTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.353458414 (183h53m3.033653983s) [noise=true , address=77.90.31.14:4661 , piece_id=GLHSZYEWG577XHKA6IP7BANUPLJYBBBZL6GYCFBPRP35FREHMWAA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:06.618637531 (183h53m3.2988331s) [noise=true , address=45.91.116.13:4154 , piece_id=XR3VHPMSA2TNWMYG5CJV6DBPOOW3E4XZTZXOYTSC3SXNL6YPCUTA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.074977606 (183h53m4.755173175s) [noise=true , address=45.91.116.6:4975 , piece_id=FYIANK4VEDILAH36C3NHEWLG7OGACRAWCYPWYBFCQJZEF5AG33BA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:08.193455912 (183h53m4.873651481s) [noise=true , address=45.91.116.12:4225 , piece_id=IMHTCE4ZQ7EO3UHRSBIZ4NKZWVTNRHREEQDPO5GSQ6XJWIWOCZEQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.606231531 (183h53m6.2864271s) [noise=true , address=77.90.31.10:4517 , piece_id=M5F435XLKIWJOJ7MWIUP2DB5GQE4GXTSQFIENDLBNA772VBHX5ZQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:09.729210390 (183h53m6.409405959s) [noise=true , address=45.91.116.10:4081 , piece_id=CLXBGRY3FHXEZMONVIFEWY52OZ3U2J6T6AAHFXQXSRSPVNLFSZCA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:12.441592321 (183h53m9.12178789s) [noise=true , address=77.90.31.16:4733 , piece_id=7ONHWYVSVQO7NLUEZBMDAVOMN7BFFD56QOC3RNHLMQZ6FQ5MJTRA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:17.884919977 (183h53m14.565115547s) [noise=true , address=77.90.31.10:4519 , piece_id=N2XG3DND64W2SSUM7XU6N3M7TEYLJOUBT73K7TQ6COEBRKFFIWVA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.848711975 (183h53m16.528907544s) [noise=true , address=77.90.31.16:4735 , piece_id=7EUTX6TEXCVGFHD6BWYAULZWHZBCZJMQTTLIYUSVEWXAGKEQICQQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:19.979812743 (183h53m16.660008312s) [noise=true , address=77.90.31.13:4628 , piece_id=GVKPHXMCG4SQLXZGKMQIUT6MJKR5FRUVQDIIWPJG627TQUN5YXFA, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:22.837087414 (183h53m19.517282983s) [noise=true , address=77.90.31.21:4878 , piece_id=E7KU3KE3KCEVZCWAFETIR4I2PYRFCGM56BGU4OJVRQWG36GEE3GQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: <nil> Sep 6 06:01:24.061391504 (183h53m20.741587073s) [noise=true , address=45.91.116.12:4227 , piece_id=QT4APHZVPF27BZ5CTUXHLJQM4KEKGQKTVRL75DUYL7XVTTIHQN2Q, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46256->45.91.116.12:4227: use of closed network connection Sep 6 06:01:24.061448052 (183h53m20.741643621s) [noise=true , address=45.91.116.13:4155 , piece_id=OMOYRTB5B3J2ROPCPXWUIA4ZFKBSY2FCK6BIYFL2EVEETCEW65OQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45096->45.91.116.13:4155: use of closed network connection Sep 6 06:01:24.061491345 (183h53m20.741686914s) [noise=true , address=45.91.116.9:4263 , piece_id=T4YBCIKW2EIN6TAUO5F4HUMCYXVD5DL4SE4L3TZCKXW7IWK6YNGQ, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47816->45.91.116.9:4263: use of closed network connection Sep 6 06:01:24.061521442 (183h53m20.741717011s) [noise=true , address=45.91.116.10:4082 , piece_id=5SKCNCVERKUTA5ZCQOCZRCUR4VDDOYANVENSYR4O2EF2DGXBTV4A, seg_pos={0 0}, upload=161812]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51926->45.91.116.10:4082: use of closed network connection Sep 6 06:01:24.061539307 (183h53m20.741734876s) [seg_pos={0 0}, upload=161812]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 6 06:01:25.813674082 (183h53m22.493869652s) [upload=161812]: Done uploading segments... Sep 6 06:01:25.813704481 (183h53m22.49390007s) [upload=161812]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.427415927s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.964102803s FAIL (2025-09-06 06:04:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.964119354s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)