Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X...X.......................................................] duration: 6.224290325s FAIL (2025-10-07 01:39:25 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.224350319s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XX...X......................................................] duration: 6.290112503s FAIL (2025-10-07 01:39:04 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.290151808s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X...X..X.XXX................................................] duration: 5.954109855s FAIL (2025-10-07 01:38:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 5.954129212s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXX...X.....................................................] duration: 6.240169595s FAIL (2025-10-07 01:38:22 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.240223609s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XX...X..X.XXX...............................................] duration: 6.060749476s FAIL (2025-10-07 01:37:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.060781406s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XXX...X....................................................] duration: 6.252222313s FAIL (2025-10-07 01:37:23 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.252260114s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.XX...X..X.XXX..............................................] duration: 6.034022542s FAIL (2025-10-07 01:37:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.034059864s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.XXX...X...................................................] duration: 6.310675498s FAIL (2025-10-07 01:36:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.310702981s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..XX...X..X.XXX.............................................] duration: 6.027151679s FAIL (2025-10-07 01:36:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.027187356s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.XXX...X..................................................] duration: 6.202958276s FAIL (2025-10-07 01:36:04 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.202994534s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X..XX...X..X.XXX............................................] duration: 5.951503511s FAIL (2025-10-07 01:35:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 5.951536874s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XX..XX...X..X.XXX...........................................] duration: 6.115938426s FAIL (2025-10-07 01:35:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.115968572s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.XXX...X.................................................] duration: 6.031271272s FAIL (2025-10-07 01:34:57 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.031323963s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.889448246s FAIL (2025-10-07 01:34:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.889494553s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.XX..XX...X..X.XXX..........................................] duration: 6.109229009s FAIL (2025-10-07 01:34:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.109322606s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X.XXX...X................................................] duration: 6.354273707s FAIL (2025-10-07 01:33:59 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.354310066s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.73855112s FAIL (2025-10-07 01:33:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.738602939s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.917480057s FAIL (2025-10-07 01:33:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.917513991s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.866242779s FAIL (2025-10-07 01:33:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.866278137s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..XX..XX...X..X.XXX.........................................] duration: 6.063632211s FAIL (2025-10-07 01:33:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.063670405s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X.XXX...X...............................................] duration: 6.160562556s FAIL (2025-10-07 01:32:57 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.160597323s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X..XX..XX...X..X.XXX........................................] duration: 5.905399598s FAIL (2025-10-07 01:32:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 5.905433143s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.112563869s FAIL (2025-10-07 01:32:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 6.112596701s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X.XXX...X..............................................] duration: 6.175615319s FAIL (2025-10-07 01:32:21 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.175650787s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.X..XX..XX...X..X.XXX.......................................] duration: 6.13950704s FAIL (2025-10-07 01:31:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.1395389s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X.XXX...X.............................................] duration: 6.107414735s FAIL (2025-10-07 01:31:46 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.107451084s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.815528853s FAIL (2025-10-07 01:31:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.815563227s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..X..XX..XX...X..X.XXX......................................] duration: 6.100566944s FAIL (2025-10-07 01:31:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.100593655s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X.XXX...X............................................] duration: 6.139896416s FAIL (2025-10-07 01:30:59 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.139928588s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...X..XX..XX...X..X.XXX.....................................] duration: 5.964178976s FAIL (2025-10-07 01:30:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 5.96421221s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.949501327s FAIL (2025-10-07 01:30:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.949533179s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X..XX..XX...X..X.XXX....................................] duration: 6.055024954s FAIL (2025-10-07 01:30:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.055106179s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [........X.XXX...X...........................................] duration: 6.504383885s FAIL (2025-10-07 01:29:52 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.504416566s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.754979463s FAIL (2025-10-07 01:29:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.755021423s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....X..XX..XX...X..X.XXX...................................] duration: 5.912865038s FAIL (2025-10-07 01:29:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 5.912895615s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........X.XXX...X..........................................] duration: 6.620712605s FAIL (2025-10-07 01:29:13 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.620748904s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [......X..XX..XX...X..X.XXX..................................] duration: 5.928700809s FAIL (2025-10-07 01:28:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 5.928733382s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.752036457s FAIL (2025-10-07 01:28:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.75206967s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.868457705s FAIL (2025-10-07 01:28:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.868489224s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X.XXX...X.........................................] duration: 6.135197286s FAIL (2025-10-07 01:28:13 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.135226642s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......X..XX..XX...X..X.XXX.................................] duration: 6.005864602s FAIL (2025-10-07 01:28:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.005899478s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.76452664s FAIL (2025-10-07 01:27:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.764557771s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........X..XX..XX...X..X.XXX................................] duration: 5.893288619s FAIL (2025-10-07 01:27:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 5.893339597s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X.XXX...X........................................] duration: 6.303195706s FAIL (2025-10-07 01:27:01 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.303232364s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.........X..XX..XX...X..X.XXX...............................] duration: 6.091307282s FAIL (2025-10-07 01:27:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 6.091334815s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.794353641s FAIL (2025-10-07 01:26:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 00:49:11.270477690 (323h17m56.732570163s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.271660442 (323h17m56.733752925s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272073198 (323h17m56.734165681s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.272607475 (323h17m56.734699968s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273069225 (323h17m56.735161698s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273469988 (323h17m56.735562471s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.273935304 (323h17m56.736027787s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.274462949 (323h17m56.736555432s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.275071137 (323h17m56.73716361s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.277843135 (323h17m56.739935619s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.279164522 (323h17m56.741256995s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.282786378 (323h17m56.744878851s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284392847 (323h17m56.74648533s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.284956089 (323h17m56.747048562s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.286010447 (323h17m56.74810292s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287033115 (323h17m56.749143441s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:11.287555179 (323h17m56.749647662s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Uploading piece... Oct 7 00:49:12.734127766 (323h17m58.19622025s) [upload=284525]: Next returned nil segment Oct 7 00:49:12.734149448 (323h17m58.196241921s) [upload=284525]: Waiting for error group managing segments... Oct 7 00:49:13.140405618 (323h17m58.602498101s) [noise=true , address=145.79.227.23:4566 , piece_id=R7Y4CG2LDCK4MDNV2JHG5Z66CPZFDRK5PZV6FEHYDAWWLLDKIFWQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.140737971 (323h17m58.602830454s) [noise=true , address=145.79.227.16:4919 , piece_id=IVTVRCHBJAMWAO4RB2WYU7JNBEDNF3RCTMPX5BZJ3DWMU43R2Q5A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.147417470 (323h17m58.609509953s) [noise=true , address=145.79.227.31:4813 , piece_id=YZUK53DVVYXC3SQNY2CMZUYJCLIRDF5YK2MKW2H5HNYVNRL33BMA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.159901651 (323h17m58.621994124s) [noise=true , address=145.79.227.27:4694 , piece_id=3DMPF2HRM5AATOTNKOMLHVF2H62M5GNEPZD36PKOB7JOXYSQQLGQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.164472215 (323h17m58.626564698s) [noise=true , address=145.79.227.22:4514 , piece_id=J5S6DVBKN5TSSAC66IYQLNW4IDBQQ7G6QPBU4U723DYUAWCEKRTQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.264687051 (323h17m58.726779534s) [noise=true , address=45.91.116.13:4169 , piece_id=XIJA6BU3BVA3WS3RFNPMMLUXMCFVSLDSQTMQG7INJE6IQAMCKF6Q, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.267171182 (323h17m58.729263655s) [noise=true , address=77.90.31.10:4542 , piece_id=ABVXFVZOWDY6KKNEJMM3AYS4WM5E77JDZOYGWJ43Q4BBCMWEKZNQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.272807604 (323h17m58.734900087s) [noise=true , address=45.91.116.4:4943 , piece_id=DGVZZHGPKN2TQ37TR2UWXEZMLGJLVZEQL3SI5C2W26K7BLUSLDPQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.278449096 (323h17m58.740541569s) [noise=true , address=77.90.31.21:4896 , piece_id=YRZ6T2CRK5NPIE532I23AHVYGHO2MCVC6D42LA72USWW3NZLEZQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.299289319 (323h17m58.761381792s) [noise=true , address=45.91.116.12:4249 , piece_id=F5E65FG3DESXPVPE33377JXUFYRNJMNDGGL6HX77NBVTJPZIVXAA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.311651950 (323h17m58.773744423s) [noise=true , address=45.91.116.5:4303 , piece_id=RXH6G3NNH3HTBDSJ2OKXIBUZBSD6QNRVVODJVLIFXKT4QUWTDRGA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.323896927 (323h17m58.7859894s) [noise=true , address=77.90.31.15:4718 , piece_id=AJOIU3PWFOMNZOROFECJUX2SQYZPNXHNNDGSZ44KTPP4N2FIIIDQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.361671816 (323h17m58.823764299s) [noise=true , address=77.90.31.12:4601 , piece_id=JQOQKYYV6X5M7FL43KCPSED5IF5JK4GUQWHCRJRBVQ22PYQLRNIQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364388519 (323h17m58.826481002s) [noise=true , address=45.91.116.9:4277 , piece_id=K6DCOVH57MA77REORPN6IYYXHSCMUAJ2UHUBEKFBHVSEPPNSM5KQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: <nil> Oct 7 00:49:13.364415841 (323h17m58.826508324s) [seg_pos={0 0}, upload=284525]: Segment reached optimal threshold of 14 pieces. Oct 7 00:49:13.364650007 (323h17m58.82674249s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yBCEYPaZNi4HA8Kt6JPgnAtZ4FCkrEs28yTknjmBpG6A6wCTx): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364659244 (323h17m58.826751727s) [noise=true , address=77.90.31.17:4789 , piece_id=6PTPQ4BQNCB7GHVJTH374W3DCLZWUUS2OBKFFDDY6NLZC5MACK6A, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364690343 (323h17m58.826782847s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jN6HbQbPBjk645swQd33LrTfb3LbAWk1dPNXU45zFrgmm6Q82): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364702036 (323h17m58.826794519s) [noise=true , address=45.91.116.8:4068 , piece_id=BVE7LSHTR2AQ24X5HQ6HVMRQIWJVZLGIG5DPVHTJH3VVDGBA2BVQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364734859 (323h17m58.826827352s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UVCFqkZwuXQ28GdfQahRcC5jwfWhoanWdTs7Z1jdHkxVVDARi): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364741732 (323h17m58.826834215s) [noise=true , address=77.90.31.19:4852 , piece_id=JHUSGCTYLD7SMQKFC3CNU6URQMFEZEC53QNWQCL2ERFMEWSQVHSQ, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364793700 (323h17m58.826886173s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NFW5tt5R84RaTvY9Up75kNjZ7bmLAgUun41KLdW9Q7eMv27Sq): protocol: expected piece hash; context canceled; EOF Oct 7 00:49:13.364801255 (323h17m58.826893728s) [noise=true , address=77.90.31.11:4570 , piece_id=2S6KTNFM27XDGRGJKUJCAK7AVBUXYYIOIR52DQ5SU4B7GFDEMBQA, seg_pos={0 0}, upload=284525]: Long tail cancellation due to optimal threshold Oct 7 00:49:13.364821233 (323h17m58.826913716s) [seg_pos={0 0}, upload=284525]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 00:49:13.364916545 (323h17m58.827009028s) [upload=284525]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 00:49:13.364924219 (323h17m58.827016702s) [upload=284525]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 00:49:32.774142630 (323h18m18.236235103s) [upload=284525]: Done uploading segments... Oct 7 00:49:32.774167727 (323h18m18.2362602s) [upload=284525]: Upload finished. err: <nil> Duration: 5.794384869s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [............X.XXX...X.......................................] duration: 6.479725178s FAIL (2025-10-07 01:26:22 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.479786525s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..........X..XX..XX...X..X.XXX..............................] duration: 5.926103343s FAIL (2025-10-07 01:26:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 5.926235123s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...........X..XX..XX...X..X.XXX.............................] duration: 5.938062556s FAIL (2025-10-07 01:25:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, Piece ID: TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q) context deadline exceeded ======LOGS====== t 7 01:03:50.459807798 (323h32m35.921900281s) [upload=284743]: Starting upload Oct 7 01:03:50.459867762 (323h32m35.921960235s) [upload=284743]: Done starting upload Oct 7 01:03:50.460065419 (323h32m35.922157892s) [upload=284743]: Uploading segments... Oct 7 01:03:50.460363927 (323h32m35.92245642s) [upload=284743]: Got next segment. Inline: false Oct 7 01:03:50.460404864 (323h32m35.922497347s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 01:03:50.460410506 (323h32m35.922502979s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 01:03:50.715813017 (323h32m36.17790551s) [seg_pos={0 0}, upload=284743]: Begin upload segment... Oct 7 01:03:50.715899621 (323h32m36.177992094s) [seg_pos={0 0}, upload=284743]: Done begin upload segment. Oct 7 01:03:50.715924098 (323h32m36.178016571s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.716809483 (323h32m36.178901957s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717243621 (323h32m36.179336094s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717354882 (323h32m36.179447365s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717620609 (323h32m36.179713082s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717632671 (323h32m36.179725144s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.717959705 (323h32m36.180052178s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718063302 (323h32m36.180155785s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718280355 (323h32m36.180372838s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718461950 (323h32m36.180554434s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718574995 (323h32m36.180667479s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718823789 (323h32m36.180916272s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.718828328 (323h32m36.180920801s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719188033 (323h32m36.181280516s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.719709937 (323h32m36.18180242s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:50.720907838 (323h32m36.183000321s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Uploading piece... Oct 7 01:03:52.431962504 (323h32m37.894054977s) [upload=284743]: Next returned nil segment Oct 7 01:03:52.431988694 (323h32m37.894081167s) [upload=284743]: Waiting for error group managing segments... Oct 7 01:03:52.719188280 (323h32m38.181280753s) [noise=true , address=145.79.227.2:4114 , piece_id=ARFXWCGWCWSRVDYBZKI63TP2DIOA32T4VQ52XVEEO7GCA5777OAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.719818340 (323h32m38.181910823s) [noise=true , address=145.79.227.16:4910 , piece_id=SRA6ZX7QUQAXDSP6WYM6SGSXSITYIHMHMLFTIPCQEXWFVRLRQWCQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.720408995 (323h32m38.182501478s) [noise=true , address=145.79.227.13:4804 , piece_id=RKEXEBQOV27CJUN2IRI3A2JMFC3AQJME6FIYMEYGWPKGKMJEOMJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.727389888 (323h32m38.189482361s) [noise=true , address=145.79.227.2:4113 , piece_id=32ST7UU6NNWBQGMFOUWVLUL44T436GY4RZRQPQCHQP35WVNWNMWQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.729275599 (323h32m38.191368072s) [noise=true , address=145.79.227.4:4187 , piece_id=O2BLXFNRNNEGN3I4P3NJZF365MBVKF2N32G35QLMYO3UIB53FHOQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.936259661 (323h32m38.398352144s) [noise=true , address=45.91.116.6:4973 , piece_id=2SDCBRJRLQPZWVDP5MXFVJ6C2WKX3HRRQ5QVUFY2FVMIJMLK3U3Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.987763733 (323h32m38.449856206s) [noise=true , address=77.90.31.15:4697 , piece_id=2U7GVBUKGXDAHV2QMQR5BTF7JDS5URTGZI2WVNACP6SQM6DWXWAA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.991511610 (323h32m38.453604093s) [noise=true , address=45.91.116.4:4938 , piece_id=DOGSDCSN2IKFISSZJ7VJQUZCASUYWUWHINLVBHO3HM5TXT4MLLHQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:52.996182033 (323h32m38.458274506s) [noise=true , address=45.91.116.12:4225 , piece_id=TERMFVWD5J6NBB6GTQWEPQAFP4WOJ6CZKV7BX633DQHSJIHEBDKA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.006584812 (323h32m38.468677295s) [noise=true , address=77.90.31.13:4626 , piece_id=7KVEAF7N57KNIUAV5OK3XGBXANPMY3VO3VUNVQ2HJOOMTZANVUJA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.059084270 (323h32m38.521176753s) [noise=true , address=77.90.31.17:4771 , piece_id=ZGDQH7ADAYHD4JVDFNCEO4DCD6JUGK6IPIJAHRNVSMESYGKYHNLA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.067772633 (323h32m38.529865117s) [noise=true , address=45.91.116.5:4299 , piece_id=N5C4XJ5HVMJRKECKCJPVTYSQXQP3KLRHMNYIAGC2M2CG6N25G5XA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091738148 (323h32m38.553830631s) [noise=true , address=45.91.116.14:4190 , piece_id=3IIOMD2FG6WQQ4SGF6RAAISUW4I4XPOLCZUTLPOXB6BNMYKIS26Q, seg_pos={0 0}, upload=284743]: Done uploading piece. err: <nil> Oct 7 01:03:53.091767073 (323h32m38.553859546s) [seg_pos={0 0}, upload=284743]: Segment reached optimal threshold of 13 pieces. Oct 7 01:03:53.091898233 (323h32m38.553990716s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091909434 (323h32m38.554001917s) [noise=true , address=45.91.116.6:4974 , piece_id=M4JHTDMJB5SM2XZPBOS3NPTJ3RJFNPBP344LZA3FT4S4JF3AAA7A, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.091994296 (323h32m38.554086769s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 7 01:03:53.091999285 (323h32m38.554091758s) [noise=true , address=45.91.116.9:4263 , piece_id=4F5ZH6FHPBPTDFKH5YMYVDCOXDDQSYQGN4E65XNZ6GZIWTT6H5GQ, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092009454 (323h32m38.554101937s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48488->45.91.116.7:4009: use of closed network connection Oct 7 01:03:53.092021548 (323h32m38.554114021s) [noise=true , address=45.91.116.7:4009 , piece_id=3SIGONZVXA3AQKHQAQLJORCLN3REBC6EBJOEAPYMFJ3G7E22PEDA, seg_pos={0 0}, upload=284743]: Long tail cancellation due to optimal threshold Oct 7 01:03:53.092037167 (323h32m38.55412964s) [seg_pos={0 0}, upload=284743]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 01:03:53.092099667 (323h32m38.55419214s) [upload=284743]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 01:03:53.092104275 (323h32m38.554196748s) [upload=284743]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 01:03:53.458903252 (323h32m38.920995725s) [upload=284743]: Done uploading segments... Oct 7 01:03:53.458925414 (323h32m38.921017887s) [upload=284743]: Upload finished. err: <nil> Duration: 5.938093596s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............X.XXX...X......................................] duration: 6.114215163s FAIL (2025-10-07 01:25:38 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.114251563s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)