Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.837578754s FAIL (2025-07-26 20:35:38 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.837605674s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................................XXX.........] duration: 6.339233621s FAIL (2025-07-26 20:35:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.339254721s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X..............X.............X..........] duration: 5.948000284s FAIL (2025-07-26 20:35:10 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.948026585s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 6.087243697s FAIL (2025-07-26 20:34:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.08726051s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................XXX........] duration: 6.600127274s FAIL (2025-07-26 20:34:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.600147692s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.907275173s FAIL (2025-07-26 20:34:42 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.907302886s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X..............X.............X.........] duration: 5.83619558s FAIL (2025-07-26 20:34:41 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.836251786s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XXX.......] duration: 6.040679446s FAIL (2025-07-26 20:34:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.040709743s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.943216758s FAIL (2025-07-26 20:34:15 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.943246096s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................XXX......] duration: 6.282496114s FAIL (2025-07-26 20:33:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.282523727s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.922676418s FAIL (2025-07-26 20:33:52 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.922710424s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 6.215507292s FAIL (2025-07-26 20:33:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.21553227s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................XXX.....] duration: 6.040052105s FAIL (2025-07-26 20:33:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.040074727s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X..............X.............X........] duration: 6.401086568s FAIL (2025-07-26 20:33:28 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.401111777s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 6.315415676s FAIL (2025-07-26 20:33:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.315439302s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................XXX....] duration: 6.297646549s FAIL (2025-07-26 20:33:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.297667069s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X..............X.............X.......] duration: 6.123044348s FAIL (2025-07-26 20:33:02 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.123069195s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.902858242s FAIL (2025-07-26 20:32:58 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.902911783s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X..............X.............X......] duration: 6.688250631s FAIL (2025-07-26 20:32:32 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.688276381s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.890045115s FAIL (2025-07-26 20:32:23 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.890072979s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 5.786847079s FAIL (2025-07-26 20:32:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.786868199s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................................XXX...] duration: 6.666776276s FAIL (2025-07-26 20:32:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.66679957s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X..............X.............X.....] duration: 6.433309634s FAIL (2025-07-26 20:32:02 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.433355673s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.849086781s FAIL (2025-07-26 20:31:41 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.849116077s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.021847452s FAIL (2025-07-26 20:31:24 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 2.021879264s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XXX..] duration: 6.276648397s FAIL (2025-07-26 20:31:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.276678674s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 6.063746161s FAIL (2025-07-26 20:31:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.063774825s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.889467741s FAIL (2025-07-26 20:30:53 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.889494713s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X..............X.............X....] duration: 6.488068642s FAIL (2025-07-26 20:30:51 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.488103218s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 6.12774504s FAIL (2025-07-26 20:30:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.127766983s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X..............X.............X...] duration: 6.413757552s FAIL (2025-07-26 20:30:21 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.413782078s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XXX.] duration: 6.241278687s FAIL (2025-07-26 20:30:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.241312021s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.073412209s FAIL (2025-07-26 20:30:05 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 2.073442327s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 6.191737627s FAIL (2025-07-26 20:30:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.191767874s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X..............X.............X..] duration: 8.428737797s FAIL (2025-07-26 20:29:26 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 8.428754309s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.230004058s FAIL (2025-07-26 20:29:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.230030889s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X..............X.............X.] duration: 21.800475871s FAIL (2025-07-26 20:29:18 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 21.800502661s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.047792227s FAIL (2025-07-26 20:29:18 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 2.047831222s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 23.920052727s FAIL (2025-07-26 20:29:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.920068707s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1.600679152s FAIL (2025-07-26 20:29:01 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.600729248s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000476169s FAIL (2025-07-26 20:28:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000985864s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.234080195s FAIL (2025-07-26 20:28:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ctBegin Jul 26 20:28:17.907897729 (56h52m27.630460759s) [upload=54593]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:18.359887517 (56h52m28.082450557s) [seg_pos={0 0}, upload=54593]: Begin upload segment... Jul 26 20:28:18.359930629 (56h52m28.082493659s) [seg_pos={0 0}, upload=54593]: Done begin upload segment. Jul 26 20:28:18.359943915 (56h52m28.082506955s) [noise=true , address=45.91.116.5:4299 , piece_id=TKO3OQY3IHVRQFXH2DHCYBLNX7KM5LRKEUNF3CIIWKNC5I2JZK5Q, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.359951779 (56h52m28.08251482s) [noise=true , address=77.90.31.16:4734 , piece_id=ZMU55WNRZQYLR2SJ6PLDTOF4GUXB7IJFHBDPFFFM72KFK5AISACA, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.360060778 (56h52m28.082623818s) [noise=true , address=77.90.31.18:4806 , piece_id=BRLSA7CROQIUEGRR7XIL2NDGWUAQQB2ZSYFFP5UL5YVJV6NP52BA, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.360332839 (56h52m28.082895879s) [noise=true , address=77.90.31.15:4699 , piece_id=VXQOIM5UI66QMSKXJR7ZPQG7QBTRIBYOP3ZTI6AGIFCWN6D44W6Q, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.360344751 (56h52m28.082907791s) [noise=true , address=45.91.116.4:4940 , piece_id=N64JQI4GZ4PCC2EFTTNO7WQG2FIM56AFKPI7XGPD3IUJ2HLSDJYQ, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.360574361 (56h52m28.083137391s) [noise=true , address=45.91.116.13:4153 , piece_id=RLXWO5BLPZWCNS2D5Y6LOXPA6LNWLPDA46GNZT4IUYYSP2Z2ZKBQ, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.360798009 (56h52m28.083361049s) [noise=true , address=77.90.31.19:4841 , piece_id=2VCUX4NVYLRPDQGHWZ7QU7SHT6WRGZWTIEZWA3NDI4MYT6MQNMIA, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.361038209 (56h52m28.083601239s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.361255254 (56h52m28.083818294s) [noise=true , address=104.36.82.14:4221 , piece_id=SYZPGQGLLZ5Y7JBAYHUSFNEYPLGKFV25O4T5RRNBF7LK2VLLR6NA, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.361494030 (56h52m28.08405707s) [noise=true , address=45.91.116.9:4262 , piece_id=FXNKH2BTMYQIJ7HIZ7O6P7LBVKJTQKKUUXVS4H4JLCNZEKEX7MGA, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.361679225 (56h52m28.084242265s) [noise=true , address=45.91.116.12:4227 , piece_id=LMYT5T7JO7MDZ2GKJGB7JTBJ7WYJ2NET36WUIIMNGF4HNUW3TQJQ, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.361885799 (56h52m28.08444884s) [noise=true , address=104.36.82.11:4115 , piece_id=RPZMJC4N55WTDS7LEK6NDTX5MWJ6SACBYU7PWOGWFK3JMGNU23SQ, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.362079411 (56h52m28.084642451s) [noise=true , address=45.91.116.11:4119 , piece_id=67BYVSDLTORIUSC26AFYAGEUU6UJVFAUO7HNB7M46LMQ5VEGWWJA, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.362263903 (56h52m28.084826933s) [noise=true , address=45.91.116.13:4155 , piece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.362472873 (56h52m28.085035913s) [noise=true , address=45.91.116.8:4048 , piece_id=MQ4KMWTFK3EU7LM6YSCTGEL3PX7YUJQ657XPO4IHICCJ3356U7KA, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:18.362656916 (56h52m28.085219956s) [noise=true , address=45.91.116.9:4261 , piece_id=A4B3CU2TRN3GXEPCRL4NPLACMGMGEJQJ22SNWBGSTQS6ELVICJHQ, seg_pos={0 0}, upload=54593]: Uploading piece... Jul 26 20:28:20.140805391 (56h52m29.863368421s) [upload=54593]: Next returned nil segment Jul 26 20:28:20.140824618 (56h52m29.863387658s) [upload=54593]: Waiting for error group managing segments... Jul 26 20:28:20.426147701 (56h52m30.148710731s) [noise=true , address=77.90.31.15:4699 , piece_id=VXQOIM5UI66QMSKXJR7ZPQG7QBTRIBYOP3ZTI6AGIFCWN6D44W6Q, seg_pos={0 0}, upload=54593]: Done uploading piece. err: <nil> Jul 26 20:28:20.455182474 (56h52m30.177745514s) [noise=true , address=77.90.31.19:4841 , piece_id=2VCUX4NVYLRPDQGHWZ7QU7SHT6WRGZWTIEZWA3NDI4MYT6MQNMIA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: <nil> Jul 26 20:28:20.538055415 (56h52m30.260618455s) [noise=true , address=77.90.31.18:4806 , piece_id=BRLSA7CROQIUEGRR7XIL2NDGWUAQQB2ZSYFFP5UL5YVJV6NP52BA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: <nil> Jul 26 20:28:20.616317455 (56h52m30.338880495s) [noise=true , address=77.90.31.16:4734 , piece_id=ZMU55WNRZQYLR2SJ6PLDTOF4GUXB7IJFHBDPFFFM72KFK5AISACA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: <nil> Jul 26 20:28:20.763088624 (56h52m30.485651654s) [noise=true , address=104.36.82.14:4221 , piece_id=SYZPGQGLLZ5Y7JBAYHUSFNEYPLGKFV25O4T5RRNBF7LK2VLLR6NA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: <nil> Jul 26 20:28:21.388763026 (56h52m31.111326056s) [noise=true , address=104.36.82.11:4115 , piece_id=RPZMJC4N55WTDS7LEK6NDTX5MWJ6SACBYU7PWOGWFK3JMGNU23SQ, seg_pos={0 0}, upload=54593]: Done uploading piece. err: <nil> Jul 26 20:28:47.254738767 (56h52m56.977301868s) [noise=true , address=45.91.116.5:4299 , piece_id=TKO3OQY3IHVRQFXH2DHCYBLNX7KM5LRKEUNF3CIIWKNC5I2JZK5Q, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.254835232 (56h52m56.977398313s) [noise=true , address=45.91.116.13:4153 , piece_id=RLXWO5BLPZWCNS2D5Y6LOXPA6LNWLPDA46GNZT4IUYYSP2Z2ZKBQ, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.254938981 (56h52m56.977502061s) [noise=true , address=45.91.116.12:4227 , piece_id=LMYT5T7JO7MDZ2GKJGB7JTBJ7WYJ2NET36WUIIMNGF4HNUW3TQJQ, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: rpc: dial tcp 45.91.116.12:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255052007 (56h52m56.977615057s) [noise=true , address=45.91.116.9:4262 , piece_id=FXNKH2BTMYQIJ7HIZ7O6P7LBVKJTQKKUUXVS4H4JLCNZEKEX7MGA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255107884 (56h52m56.977670934s) [noise=true , address=45.91.116.8:4048 , piece_id=MQ4KMWTFK3EU7LM6YSCTGEL3PX7YUJQ657XPO4IHICCJ3356U7KA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: rpc: dial tcp 45.91.116.8:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255163270 (56h52m56.97772631s) [noise=true , address=45.91.116.4:4940 , piece_id=N64JQI4GZ4PCC2EFTTNO7WQG2FIM56AFKPI7XGPD3IUJ2HLSDJYQ, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255237512 (56h52m56.977800562s) [noise=true , address=45.91.116.11:4119 , piece_id=67BYVSDLTORIUSC26AFYAGEUU6UJVFAUO7HNB7M46LMQ5VEGWWJA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:41294->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255287908 (56h52m56.977850948s) [noise=true , address=45.91.116.9:4261 , piece_id=A4B3CU2TRN3GXEPCRL4NPLACMGMGEJQJ22SNWBGSTQS6ELVICJHQ, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255315852 (56h52m56.977878902s) [noise=true , address=45.91.116.13:4155 , piece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 30.234100354s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X..............X.............X] duration: 30.237357051s FAIL (2025-07-26 20:28:15 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.237384194s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001283022s FAIL (2025-07-26 20:28:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001304795s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.241272981s FAIL (2025-07-26 20:27:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== uestItem_ObjectCommit Jul 26 20:26:39.077293583 (56h50m48.799856623s) [upload=54566]: Done uploading segments... Jul 26 20:26:39.077312048 (56h50m48.799875088s) [upload=54566]: Upload finished. err: <nil> Jul 26 20:27:27.131361030 (56h51m36.85392406s) [upload=54578]: Starting upload Jul 26 20:27:27.131427407 (56h51m36.853990447s) [upload=54578]: Done starting upload Jul 26 20:27:27.131522619 (56h51m36.854085659s) [upload=54578]: Uploading segments... Jul 26 20:27:27.131617972 (56h51m36.854181012s) [upload=54578]: Got next segment. Inline: false Jul 26 20:27:27.131650824 (56h51m36.854213864s) [upload=54578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:27:27.131656906 (56h51m36.854219936s) [upload=54578]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:27:27.371532474 (56h51m37.094095514s) [seg_pos={0 0}, upload=54578]: Begin upload segment... Jul 26 20:27:27.371584223 (56h51m37.094147263s) [seg_pos={0 0}, upload=54578]: Done begin upload segment. Jul 26 20:27:27.371616194 (56h51m37.094179234s) [noise=true , address=45.91.116.5:4297 , piece_id=6ERPANQ4QIFS5UTMXUFIFCPAJBKAT7W6U6C5Y3YZEVR64SMUF67A, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.371656692 (56h51m37.094219732s) [noise=true , address=45.91.116.11:4119 , piece_id=2DB2DRJU3IHKYFZO5OJJ4TRVDABNFORHEPS6IRWKYMTHI2D33PRA, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.372096353 (56h51m37.094659383s) [noise=true , address=45.91.116.11:4117 , piece_id=YJHW5JVCGQHDSSEYVGJYRQ6OFHV7SVICBWMDLIJ7CN3K7NSASZ5A, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.372140037 (56h51m37.094703077s) [noise=true , address=45.91.116.13:4155 , piece_id=5NQ6GNRERHHV4QNS2AASVC5A2DIM2U6FLIUXR2DA6YVL3HS6TOCA, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.372413589 (56h51m37.09497662s) [noise=true , address=45.91.116.13:4154 , piece_id=TQHHFEUMUGKUTHXNXTUFZM735R2SUQJNU6VLUP3AHOAWXTIZHUGQ, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.372437115 (56h51m37.095000155s) [noise=true , address=104.36.82.6:4874 , piece_id=CFOFUWK7XDZTATLVFQPDYROLFZP727K5HI5AWWJ4XHRAO5G7CY3A, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.372683316 (56h51m37.095246356s) [noise=true , address=104.36.82.9:4007 , piece_id=MY3RBUP7FSH4MQ5S7ANZDP35XAKMOTZSUI3TXFMIAAK567TMRYYA, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.372748361 (56h51m37.095311401s) [noise=true , address=45.91.116.8:4047 , piece_id=NAJ5QBJ6MO2YCKN5FNOCKHCJZAX7RRZYIKBEKFLHNOOH7IBQCFIA, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.373018347 (56h51m37.095581387s) [noise=true , address=45.91.116.6:4973 , piece_id=XKND5WPIHZFSNDFXKL635GMXAJJLS3Z4BVQ4AXBMEAB76SCQ72TA, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.373057833 (56h51m37.095620873s) [noise=true , address=77.90.31.19:4843 , piece_id=N4UCUWNNFPMUDF3B4ZUUEYLGPWV2USR2NWFECC6GY2LH5XLIB6QA, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.373300978 (56h51m37.095864018s) [noise=true , address=104.36.82.10:4078 , piece_id=QF5ZEQQAIPYEMUAMH33F4PBESVNCIRB2XBMTAGZQOVZBXZ5HWLDA, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.373384919 (56h51m37.095947959s) [noise=true , address=77.90.31.11:4555 , piece_id=2EWZ4OU7OMTDO52H36SFBPL4ZXWU2E565U5NB6FL7TPOGR4LJR6A, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.373508595 (56h51m37.096071635s) [noise=true , address=104.36.82.14:4221 , piece_id=RJOCMJ372V3YKK3VZ4LICQNMCV6D53N5RQ5RBJ2NFTXG5HZUOOKA, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.373678880 (56h51m37.09624192s) [noise=true , address=45.91.116.7:4011 , piece_id=H6EOR6QFSYUFKXUTOKP2ZE4VQ3CR4ZYFWZERPDYQ7GNU5F7EOSOQ, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.374009082 (56h51m37.096572122s) [noise=true , address=104.36.82.16:4295 , piece_id=6VTZEIQ3B24F7Y6XXVPBOAR2LDIO3VTNVUVYAQSNT2IKOCDQNTGA, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:27.374295340 (56h51m37.09685839s) [noise=true , address=45.91.116.14:4190 , piece_id=H3PHUVHOSEH64T75OUICSL5GXVROAISTQALTSKIYNVMOBTZ2FAFA, seg_pos={0 0}, upload=54578]: Uploading piece... Jul 26 20:27:29.084642951 (56h51m38.807205981s) [upload=54578]: Next returned nil segment Jul 26 20:27:29.084704169 (56h51m38.807267219s) [upload=54578]: Waiting for error group managing segments... Jul 26 20:27:29.455161361 (56h51m39.177724391s) [noise=true , address=104.36.82.6:4874 , piece_id=CFOFUWK7XDZTATLVFQPDYROLFZP727K5HI5AWWJ4XHRAO5G7CY3A, seg_pos={0 0}, upload=54578]: Done uploading piece. err: <nil> Jul 26 20:27:29.456664046 (56h51m39.179227086s) [noise=true , address=104.36.82.16:4295 , piece_id=6VTZEIQ3B24F7Y6XXVPBOAR2LDIO3VTNVUVYAQSNT2IKOCDQNTGA, seg_pos={0 0}, upload=54578]: Done uploading piece. err: <nil> Jul 26 20:27:29.492228361 (56h51m39.214791401s) [noise=true , address=104.36.82.10:4078 , piece_id=QF5ZEQQAIPYEMUAMH33F4PBESVNCIRB2XBMTAGZQOVZBXZ5HWLDA, seg_pos={0 0}, upload=54578]: Done uploading piece. err: <nil> Jul 26 20:27:29.545864555 (56h51m39.268427595s) [noise=true , address=104.36.82.9:4007 , piece_id=MY3RBUP7FSH4MQ5S7ANZDP35XAKMOTZSUI3TXFMIAAK567TMRYYA, seg_pos={0 0}, upload=54578]: Done uploading piece. err: <nil> Jul 26 20:27:29.683189700 (56h51m39.405752741s) [noise=true , address=104.36.82.14:4221 , piece_id=RJOCMJ372V3YKK3VZ4LICQNMCV6D53N5RQ5RBJ2NFTXG5HZUOOKA, seg_pos={0 0}, upload=54578]: Done uploading piece. err: <nil> Jul 26 20:27:41.537176865 (56h51m51.259739905s) [noise=true , address=77.90.31.11:4555 , piece_id=2EWZ4OU7OMTDO52H36SFBPL4ZXWU2E565U5NB6FL7TPOGR4LJR6A, seg_pos={0 0}, upload=54578]: Done uploading piece. err: <nil> Jul 26 20:27:56.484565848 (56h52m6.207128888s) [noise=true , address=45.91.116.11:4117 , piece_id=YJHW5JVCGQHDSSEYVGJYRQ6OFHV7SVICBWMDLIJ7CN3K7NSASZ5A, seg_pos={0 0}, upload=54578]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 20:27:56.484609040 (56h52m6.20717208s) [noise=true , address=45.91.116.7:4011 , piece_id=H6EOR6QFSYUFKXUTOKP2ZE4VQ3CR4ZYFWZERPDYQ7GNU5F7EOSOQ, seg_pos={0 0}, upload=54578]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 20:27:56.484623258 (56h52m6.207186298s) [noise=true , address=45.91.116.13:4155 , piece_id=5NQ6GNRERHHV4QNS2AASVC5A2DIM2U6FLIUXR2DA6YVL3HS6TOCA, seg_pos={0 0}, upload=54578]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 20:27:56.484650079 (56h52m6.207213109s) [noise=true , address=45.91.116.13:4154 , piece_id=TQHHFEUMUGKUTHXNXTUFZM735R2SUQJNU6VLUP3AHOAWXTIZHUGQ, seg_pos={0 0}, upload=54578]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 20:27:56.484664516 (56h52m6.207227556s) [noise=true , address=45.91.116.8:4047 , piece_id=NAJ5QBJ6MO2YCKN5FNOCKHCJZAX7RRZYIKBEKFLHNOOH7IBQCFIA, seg_pos={0 0}, upload=54578]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 20:27:56.484680547 (56h52m6.207243587s) [noise=true , address=45.91.116.6:4973 , piece_id=XKND5WPIHZFSNDFXKL635GMXAJJLS3Z4BVQ4AXBMEAB76SCQ72TA, seg_pos={0 0}, upload=54578]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 20:27:56.484700114 (56h52m6.207263144s) [noise=true , address=77.90.31.19:4843 , piece_id=N4UCUWNNFPMUDF3B4ZUUEYLGPWV2USR2NWFECC6GY2LH5XLIB6QA, seg_pos={0 0}, upload=54578]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 20:27:56.484719461 (56h52m6.207282501s) [noise=true , address=45.91.116.14:4190 , piece_id=H3PHUVHOSEH64T75OUICSL5GXVROAISTQALTSKIYNVMOBTZ2FAFA, seg_pos={0 0}, upload=54578]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 20:27:56.484738859 (56h52m6.207301899s) [noise=true , address=45.91.116.11:4119 , piece_id=2DB2DRJU3IHKYFZO5OJJ4TRVDABNFORHEPS6IRWKYMTHI2D33PRA, seg_pos={0 0}, upload=54578]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 20:27:56.484752044 (56h52m6.207315074s) [noise=true , address=45.91.116.5:4297 , piece_id=6ERPANQ4QIFS5UTMXUFIFCPAJBKAT7W6U6C5Y3YZEVR64SMUF67A, seg_pos={0 0}, upload=54578]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 20:27:56.484772573 (56h52m6.207335613s) [seg_pos={0 0}, upload=54578]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:27:56.724303941 (56h52m6.446866991s) [upload=54578]: Done uploading segments... Jul 26 20:27:56.724335872 (56h52m6.446898912s) [upload=54578]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.241303841s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X..............X.............] duration: 5.786232208s FAIL (2025-07-26 20:27:09 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 5.786250122s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X..............X............] duration: 5.808169332s FAIL (2025-07-26 20:26:33 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 5.80819988s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X..............X...........] duration: 5.827253224s FAIL (2025-07-26 20:25:24 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 5.827280167s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.889665475s FAIL (2025-07-26 20:24:42 UTC) orbiter warning: uplink: too many requests Duration: 1.889691095s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X..............X..........] duration: 5.891790015s FAIL (2025-07-26 20:24:32 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 5.891819682s core uplink us1 vivint

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)