Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [............X....X.X....................X.........X........X] duration:
4.628580151s
FAIL (2025-08-23 14:59:43 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:59:39.138797173 (357h33m33.569877304s) [upload=371917]: Starting upload
Aug 23 14:59:39.138860705 (357h33m33.569940876s) [upload=371917]: Done starting upload
Aug 23 14:59:39.138933423 (357h33m33.570013554s) [upload=371917]: Uploading segments...
Aug 23 14:59:39.139072680 (357h33m33.570152811s) [upload=371917]: Got next segment. Inline: false
Aug 23 14:59:39.139119329 (357h33m33.57019946s) [upload=371917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:59:39.139123717 (357h33m33.570203848s) [upload=371917]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:59:39.660982166 (357h33m34.092062297s) [seg_pos={0 0}, upload=371917]: Begin upload segment...
Aug 23 14:59:39.661049234 (357h33m34.092129365s) [seg_pos={0 0}, upload=371917]: Done begin upload segment.
Aug 23 14:59:39.661076115 (357h33m34.092156246s) [noise=true , address=77.90.31.10:4527 , piece_id=EM5DITJ5WYKKRPRKFCVVL5IIPXI7NDRBBTJE3WBT2BBBMDJZBJQQ, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.661099129 (357h33m34.09217926s) [noise=true , address=79.127.219.48:4103 , piece_id=UNHCBLTGRKCAIBRJJ6DA254OZB74LFKTHSLG6YKDDXAHSVOGZDDQ, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.661482303 (357h33m34.092562424s) [noise=true , address=104.36.82.10:4087 , piece_id=IR6NTWCDZVU56KMX3JMVPMIT5QDEWOXCJ6GDRUT6CL4JRFOXVILQ, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.661553038 (357h33m34.092633159s) [noise=true , address=45.91.116.4:4961 , piece_id=U5HQY7X4UR5AJOBKILZ3KT44Z7XOUEKIM5ZOPWOHDF7TWFD2CXWA, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.661835468 (357h33m34.092915599s) [noise=true , address=104.36.82.5:4862 , piece_id=X4OVZB46PLEXHJNJWYJKVY3HUIO2RUMGRUYQ33VD4KS3XKZL6NPA, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.662142225 (357h33m34.093222356s) [noise=true , address=77.90.31.16:4745 , piece_id=RS47VF3QW3CCLOTEQIANYSHSLRBA7EWNTUNUY63NIK7SMSTAC3YQ, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.662445245 (357h33m34.093525366s) [noise=true , address=104.36.82.41:4796 , piece_id=TO5LEXT2KZTPCSW3PPBMWWZIOCQ33ONQEZVGQSDWDK4S7EJSS6GQ, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.662685735 (357h33m34.093765856s) [noise=true , address=77.90.31.21:4886 , piece_id=MF7J35HTOTJDXTIVM6YVTXWVG2QRQ4FMIAH3JQ7QNAJWEW2ZROBA, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.662923771 (357h33m34.094003902s) [noise=true , address=77.90.31.19:4861 , piece_id=6ITHTJ7C7LK3DXJ5FB7BPVY3HPQMLXKEOMCMHM6EDHJ7CLJKOEPQ, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.663162908 (357h33m34.094243039s) [noise=true , address=77.90.31.18:4812 , piece_id=3S3OBDNS26GZ7LSEYOBB6DKLZMRRVKADV73PKWNE5WBZ3SMJF6VA, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.663400333 (357h33m34.094480464s) [noise=true , address=104.36.82.9:4031 , piece_id=3BKZXTDK22LEQVFLBQ44SSNIJEIU7YU5NEYONDYE2KVGP7JZBRNQ, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.663627607 (357h33m34.094707738s) [noise=true , address=45.91.116.10:4090 , piece_id=DFUJ75XJUEQTXUWK4JMBOWJKSZX2Z643PESDG7CRXVJXFGD3JCAQ, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.663845133 (357h33m34.094925264s) [noise=true , address=104.36.82.60:4368 , piece_id=5TAT2DWVAG5KYF4GM5GD2UUPQTNBUHTQCDUBCORK67UMT3ACDMNA, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.664128105 (357h33m34.095208237s) [noise=true , address=77.90.31.13:4631 , piece_id=K7PLTBVFKRJRYN2NVFECVYGF5FIFAS2MFFVTWWIMRWN6WIETWVWA, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.664370058 (357h33m34.095450269s) [noise=true , address=45.91.116.7:4039 , piece_id=Y5LFMO3PXIUC6NEO5Q5HEZXLMADZ2SW2EW6Z3MVY3IQAIAZA7QSA, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.664611339 (357h33m34.095691471s) [noise=true , address=104.36.82.23:4445 , piece_id=SDF7ISKSI32Y7EZEDBNGKSGTNSTAWPGMG5PSSYOBIBXJBTKIUA6Q, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.664828776 (357h33m34.095908907s) [noise=true , address=104.36.82.44:4882 , piece_id=YLZS2UQA7Q2TBYZB5JCTMMS26DHLUDIM2RWYHKQ32UFT4FY7IKIQ, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:39.665053746 (357h33m34.096133877s) [noise=true , address=79.127.201.215:4345 , piece_id=4KSYTORV3MTXWWU5XCKD6BOYEVCUWYOUUEKVWSHTMB7IWQJE2UGQ, seg_pos={0 0}, upload=371917]: Uploading piece...
Aug 23 14:59:41.164720620 (357h33m35.595800751s) [upload=371917]: Next returned nil segment
Aug 23 14:59:41.164737493 (357h33m35.595817624s) [upload=371917]: Waiting for error group managing segments...
Aug 23 14:59:41.550326462 (357h33m35.981406583s) [noise=true , address=79.127.219.48:4103 , piece_id=UNHCBLTGRKCAIBRJJ6DA254OZB74LFKTHSLG6YKDDXAHSVOGZDDQ, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.603100430 (357h33m36.034180561s) [noise=true , address=104.36.82.60:4368 , piece_id=5TAT2DWVAG5KYF4GM5GD2UUPQTNBUHTQCDUBCORK67UMT3ACDMNA, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.666073142 (357h33m36.097153273s) [noise=true , address=104.36.82.41:4796 , piece_id=TO5LEXT2KZTPCSW3PPBMWWZIOCQ33ONQEZVGQSDWDK4S7EJSS6GQ, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.728925305 (357h33m36.160005436s) [noise=true , address=79.127.201.215:4345 , piece_id=4KSYTORV3MTXWWU5XCKD6BOYEVCUWYOUUEKVWSHTMB7IWQJE2UGQ, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.730086386 (357h33m36.161166517s) [noise=true , address=77.90.31.13:4631 , piece_id=K7PLTBVFKRJRYN2NVFECVYGF5FIFAS2MFFVTWWIMRWN6WIETWVWA, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.771530666 (357h33m36.202610797s) [noise=true , address=104.36.82.44:4882 , piece_id=YLZS2UQA7Q2TBYZB5JCTMMS26DHLUDIM2RWYHKQ32UFT4FY7IKIQ, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.791768749 (357h33m36.22284888s) [noise=true , address=45.91.116.10:4090 , piece_id=DFUJ75XJUEQTXUWK4JMBOWJKSZX2Z643PESDG7CRXVJXFGD3JCAQ, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.823125418 (357h33m36.254205549s) [noise=true , address=104.36.82.5:4862 , piece_id=X4OVZB46PLEXHJNJWYJKVY3HUIO2RUMGRUYQ33VD4KS3XKZL6NPA, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.883069414 (357h33m36.314149546s) [noise=true , address=77.90.31.10:4527 , piece_id=EM5DITJ5WYKKRPRKFCVVL5IIPXI7NDRBBTJE3WBT2BBBMDJZBJQQ, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.888614946 (357h33m36.319695077s) [noise=true , address=45.91.116.4:4961 , piece_id=U5HQY7X4UR5AJOBKILZ3KT44Z7XOUEKIM5ZOPWOHDF7TWFD2CXWA, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.910289438 (357h33m36.341369569s) [noise=true , address=77.90.31.21:4886 , piece_id=MF7J35HTOTJDXTIVM6YVTXWVG2QRQ4FMIAH3JQ7QNAJWEW2ZROBA, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.943493070 (357h33m36.374573201s) [noise=true , address=104.36.82.23:4445 , piece_id=SDF7ISKSI32Y7EZEDBNGKSGTNSTAWPGMG5PSSYOBIBXJBTKIUA6Q, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:41.994784541 (357h33m36.425864672s) [noise=true , address=77.90.31.19:4861 , piece_id=6ITHTJ7C7LK3DXJ5FB7BPVY3HPQMLXKEOMCMHM6EDHJ7CLJKOEPQ, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:42.008674028 (357h33m36.439754159s) [noise=true , address=77.90.31.18:4812 , piece_id=3S3OBDNS26GZ7LSEYOBB6DKLZMRRVKADV73PKWNE5WBZ3SMJF6VA, seg_pos={0 0}, upload=371917]: Done uploading piece. err: <nil>
Aug 23 14:59:42.008701410 (357h33m36.439781541s) [seg_pos={0 0}, upload=371917]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:59:42.008885112 (357h33m36.439965243s) [noise=true , address=77.90.31.16:4745 , piece_id=RS47VF3QW3CCLOTEQIANYSHSLRBA7EWNTUNUY63NIK7SMSTAC3YQ, seg_pos={0 0}, upload=371917]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r): protocol: expected piece hash; context canceled; EOF
Aug 23 14:59:42.008942862 (357h33m36.440022983s) [noise=true , address=45.91.116.7:4039 , piece_id=Y5LFMO3PXIUC6NEO5Q5HEZXLMADZ2SW2EW6Z3MVY3IQAIAZA7QSA, seg_pos={0 0}, upload=371917]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16oq1Yv7JfHrg4gdHfhwuMj4UsWeQQPayKV6Pj1NY2CgJhuPr2): protocol: expected piece hash; context canceled; EOF
Aug 23 14:59:42.009021152 (357h33m36.440101273s) [noise=true , address=104.36.82.10:4087 , piece_id=IR6NTWCDZVU56KMX3JMVPMIT5QDEWOXCJ6GDRUT6CL4JRFOXVILQ, seg_pos={0 0}, upload=371917]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA): protocol: write tcp 188.166.225.170:39906->104.36.82.10:4087: use of closed network connection; write tcp 188.166.225.170:39906->104.36.82.10:4087: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39906->104.36.82.10:4087: use of closed network connection
Aug 23 14:59:42.009066298 (357h33m36.440146429s) [noise=true , address=104.36.82.9:4031 , piece_id=3BKZXTDK22LEQVFLBQ44SSNIJEIU7YU5NEYONDYE2KVGP7JZBRNQ, seg_pos={0 0}, upload=371917]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Hx9KabcKhsb2oGPefNr12E2ibDkaVnPJ6Q38rtxxudkDdXMta): protocol: write tcp 188.166.225.170:46198->104.36.82.9:4031: use of closed network connection; write tcp 188.166.225.170:46198->104.36.82.9:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->104.36.82.9:4031: use of closed network connection
Aug 23 14:59:42.009083481 (357h33m36.440163612s) [seg_pos={0 0}, upload=371917]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:59:42.009146411 (357h33m36.440226542s) [upload=371917]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:59:42.009150269 (357h33m36.4402304s) [upload=371917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:59:42.354428116 (357h33m36.785508247s) [upload=371917]: Done uploading segments...
Aug 23 14:59:42.354445359 (357h33m36.78552549s) [upload=371917]: Upload finished. err: <nil>
Duration: 4.628602013s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.............X....X.X....................X.........X........] duration:
6.021286517s
FAIL (2025-08-23 14:59:16 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 78ea70faf07caf1b60b4ca1c3d20ab82be1edff82179dffb3e7c40e6fba4e870 != before fb7a1b9a15ab61c6d2d8961ed8581d81a88ea41c13daddb60db1567d3816571c
======LOGS======
Aug 23 14:53:14.233761975 (357h27m8.664842106s) [upload=371802]: Starting upload
Aug 23 14:53:14.233969873 (357h27m8.665050004s) [upload=371802]: Done starting upload
Aug 23 14:53:14.234025399 (357h27m8.66510553s) [upload=371802]: Uploading segments...
Aug 23 14:53:14.234192869 (357h27m8.665273s) [upload=371802]: Got next segment. Inline: false
Aug 23 14:53:14.234222967 (357h27m8.665303098s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:53:14.234227315 (357h27m8.665307446s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:53:14.686493825 (357h27m9.117573956s) [seg_pos={0 0}, upload=371802]: Begin upload segment...
Aug 23 14:53:14.686550944 (357h27m9.117631075s) [seg_pos={0 0}, upload=371802]: Done begin upload segment.
Aug 23 14:53:14.686578557 (357h27m9.117658688s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687030563 (357h27m9.118110694s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687351386 (357h27m9.118431527s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687662001 (357h27m9.118742132s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687978075 (357h27m9.119058216s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688266948 (357h27m9.119347079s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688613230 (357h27m9.119693361s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688908004 (357h27m9.119988135s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689190886 (357h27m9.120271017s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689540414 (357h27m9.120620545s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690019381 (357h27m9.121099512s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690316841 (357h27m9.121396972s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690607566 (357h27m9.121687697s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690879867 (357h27m9.121959998s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691151808 (357h27m9.122231939s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691407797 (357h27m9.122487928s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691694226 (357h27m9.122774357s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.692104832 (357h27m9.123184963s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:16.187338881 (357h27m10.618419002s) [upload=371802]: Next returned nil segment
Aug 23 14:53:16.187362156 (357h27m10.618442287s) [upload=371802]: Waiting for error group managing segments...
Aug 23 14:53:16.564984571 (357h27m10.996064712s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.656767810 (357h27m11.087847941s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.673119737 (357h27m11.104199868s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.701572207 (357h27m11.132652338s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.803065146 (357h27m11.234145277s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.833099403 (357h27m11.264179534s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.834445559 (357h27m11.26552569s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.855841568 (357h27m11.286921699s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.872230768 (357h27m11.303310899s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.962297120 (357h27m11.393377241s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.983119272 (357h27m11.414199403s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.990262750 (357h27m11.421342881s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.048189307 (357h27m11.479269448s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071859558 (357h27m11.502939689s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071888494 (357h27m11.502968625s) [seg_pos={0 0}, upload=371802]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:53:17.072009154 (357h27m11.503089285s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aGm17yvKPgD3nqcQyoMVvgM3gqXDDrKPNzXJY1xf2uBdcfobS): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072168800 (357h27m11.503248931s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf): protocol: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection
Aug 23 14:53:17.072218625 (357h27m11.503298756s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection
Aug 23 14:53:17.072310891 (357h27m11.503391022s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072352371 (357h27m11.503432492s) [seg_pos={0 0}, upload=371802]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:53:17.072435190 (357h27m11.503515321s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:53:17.072440520 (357h27m11.503520651s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:53:17.414793920 (357h27m11.845874052s) [upload=371802]: Done uploading segments...
Aug 23 14:53:17.414812586 (357h27m11.845892717s) [upload=371802]: Upload finished. err: <nil>
Duration: 6.021320041s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [............X.......................X...................X...] duration:
1.919469839s
FAIL (2025-08-23 14:58:55 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 3a643a909e105b85e82a5a0738bde0b178866a88bf77126d042b98bfad6ed83d != before 7c5285e6e06c880fa5448453da287f2bb9bbb2fd371f54a13ff8f88b5c9be385 ======LOGS====== Aug 23 14:57:19.162870253 (357h31m13.593950384s) [upload=371871]: Starting upload Aug 23 14:57:19.162921210 (357h31m13.594001341s) [upload=371871]: Done starting upload Aug 23 14:57:19.162976887 (357h31m13.594057018s) [upload=371871]: Uploading segments... Aug 23 14:57:19.163018857 (357h31m13.594098988s) [upload=371871]: Got next segment. Inline: true Aug 23 14:57:19.163022904 (357h31m13.594103035s) [upload=371871]: Waiting for error group managing segments... Aug 23 14:57:19.163061539 (357h31m13.59414167s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:57:19.163066037 (357h31m13.594146158s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:57:19.163068772 (357h31m13.594148903s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:57:19.502542644 (357h31m13.933622775s) [upload=371871]: Done uploading segments... Aug 23 14:57:19.502562963 (357h31m13.933643095s) [upload=371871]: Upload finished. err: <nil>
Duration: 1.919499525s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.............X.......................X...................X..] duration:
2.07679722s
FAIL (2025-08-23 14:58:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 3a643a909e105b85e82a5a0738bde0b178866a88bf77126d042b98bfad6ed83d != before 7c5285e6e06c880fa5448453da287f2bb9bbb2fd371f54a13ff8f88b5c9be385 ======LOGS====== Aug 23 14:57:19.162870253 (357h31m13.593950384s) [upload=371871]: Starting upload Aug 23 14:57:19.162921210 (357h31m13.594001341s) [upload=371871]: Done starting upload Aug 23 14:57:19.162976887 (357h31m13.594057018s) [upload=371871]: Uploading segments... Aug 23 14:57:19.163018857 (357h31m13.594098988s) [upload=371871]: Got next segment. Inline: true Aug 23 14:57:19.163022904 (357h31m13.594103035s) [upload=371871]: Waiting for error group managing segments... Aug 23 14:57:19.163061539 (357h31m13.59414167s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:57:19.163066037 (357h31m13.594146158s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:57:19.163068772 (357h31m13.594148903s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:57:19.502542644 (357h31m13.933622775s) [upload=371871]: Done uploading segments... Aug 23 14:57:19.502562963 (357h31m13.933643095s) [upload=371871]: Upload finished. err: <nil>
Duration: 2.076895459s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..............X....X.X....................X.........X.......] duration:
6.238542325s
FAIL (2025-08-23 14:58:14 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 78ea70faf07caf1b60b4ca1c3d20ab82be1edff82179dffb3e7c40e6fba4e870 != before fb7a1b9a15ab61c6d2d8961ed8581d81a88ea41c13daddb60db1567d3816571c
======LOGS======
Aug 23 14:53:14.233761975 (357h27m8.664842106s) [upload=371802]: Starting upload
Aug 23 14:53:14.233969873 (357h27m8.665050004s) [upload=371802]: Done starting upload
Aug 23 14:53:14.234025399 (357h27m8.66510553s) [upload=371802]: Uploading segments...
Aug 23 14:53:14.234192869 (357h27m8.665273s) [upload=371802]: Got next segment. Inline: false
Aug 23 14:53:14.234222967 (357h27m8.665303098s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:53:14.234227315 (357h27m8.665307446s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:53:14.686493825 (357h27m9.117573956s) [seg_pos={0 0}, upload=371802]: Begin upload segment...
Aug 23 14:53:14.686550944 (357h27m9.117631075s) [seg_pos={0 0}, upload=371802]: Done begin upload segment.
Aug 23 14:53:14.686578557 (357h27m9.117658688s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687030563 (357h27m9.118110694s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687351386 (357h27m9.118431527s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687662001 (357h27m9.118742132s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687978075 (357h27m9.119058216s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688266948 (357h27m9.119347079s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688613230 (357h27m9.119693361s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688908004 (357h27m9.119988135s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689190886 (357h27m9.120271017s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689540414 (357h27m9.120620545s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690019381 (357h27m9.121099512s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690316841 (357h27m9.121396972s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690607566 (357h27m9.121687697s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690879867 (357h27m9.121959998s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691151808 (357h27m9.122231939s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691407797 (357h27m9.122487928s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691694226 (357h27m9.122774357s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.692104832 (357h27m9.123184963s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:16.187338881 (357h27m10.618419002s) [upload=371802]: Next returned nil segment
Aug 23 14:53:16.187362156 (357h27m10.618442287s) [upload=371802]: Waiting for error group managing segments...
Aug 23 14:53:16.564984571 (357h27m10.996064712s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.656767810 (357h27m11.087847941s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.673119737 (357h27m11.104199868s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.701572207 (357h27m11.132652338s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.803065146 (357h27m11.234145277s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.833099403 (357h27m11.264179534s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.834445559 (357h27m11.26552569s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.855841568 (357h27m11.286921699s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.872230768 (357h27m11.303310899s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.962297120 (357h27m11.393377241s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.983119272 (357h27m11.414199403s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.990262750 (357h27m11.421342881s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.048189307 (357h27m11.479269448s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071859558 (357h27m11.502939689s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071888494 (357h27m11.502968625s) [seg_pos={0 0}, upload=371802]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:53:17.072009154 (357h27m11.503089285s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aGm17yvKPgD3nqcQyoMVvgM3gqXDDrKPNzXJY1xf2uBdcfobS): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072168800 (357h27m11.503248931s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf): protocol: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection
Aug 23 14:53:17.072218625 (357h27m11.503298756s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection
Aug 23 14:53:17.072310891 (357h27m11.503391022s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072352371 (357h27m11.503432492s) [seg_pos={0 0}, upload=371802]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:53:17.072435190 (357h27m11.503515321s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:53:17.072440520 (357h27m11.503520651s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:53:17.414793920 (357h27m11.845874052s) [upload=371802]: Done uploading segments...
Aug 23 14:53:17.414812586 (357h27m11.845892717s) [upload=371802]: Upload finished. err: <nil>
Duration: 6.23857573s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..............X.......................X...................X.] duration:
2.024775652s
FAIL (2025-08-23 14:58:06 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 3a643a909e105b85e82a5a0738bde0b178866a88bf77126d042b98bfad6ed83d != before 7c5285e6e06c880fa5448453da287f2bb9bbb2fd371f54a13ff8f88b5c9be385 ======LOGS====== Aug 23 14:57:19.162870253 (357h31m13.593950384s) [upload=371871]: Starting upload Aug 23 14:57:19.162921210 (357h31m13.594001341s) [upload=371871]: Done starting upload Aug 23 14:57:19.162976887 (357h31m13.594057018s) [upload=371871]: Uploading segments... Aug 23 14:57:19.163018857 (357h31m13.594098988s) [upload=371871]: Got next segment. Inline: true Aug 23 14:57:19.163022904 (357h31m13.594103035s) [upload=371871]: Waiting for error group managing segments... Aug 23 14:57:19.163061539 (357h31m13.59414167s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:57:19.163066037 (357h31m13.594146158s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:57:19.163068772 (357h31m13.594148903s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:57:19.502542644 (357h31m13.933622775s) [upload=371871]: Done uploading segments... Aug 23 14:57:19.502562963 (357h31m13.933643095s) [upload=371871]: Upload finished. err: <nil>
Duration: 2.024805489s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...............X....X.X....................X.........X......] duration:
6.492373778s
FAIL (2025-08-23 14:57:49 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 78ea70faf07caf1b60b4ca1c3d20ab82be1edff82179dffb3e7c40e6fba4e870 != before fb7a1b9a15ab61c6d2d8961ed8581d81a88ea41c13daddb60db1567d3816571c
======LOGS======
Aug 23 14:53:14.233761975 (357h27m8.664842106s) [upload=371802]: Starting upload
Aug 23 14:53:14.233969873 (357h27m8.665050004s) [upload=371802]: Done starting upload
Aug 23 14:53:14.234025399 (357h27m8.66510553s) [upload=371802]: Uploading segments...
Aug 23 14:53:14.234192869 (357h27m8.665273s) [upload=371802]: Got next segment. Inline: false
Aug 23 14:53:14.234222967 (357h27m8.665303098s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:53:14.234227315 (357h27m8.665307446s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:53:14.686493825 (357h27m9.117573956s) [seg_pos={0 0}, upload=371802]: Begin upload segment...
Aug 23 14:53:14.686550944 (357h27m9.117631075s) [seg_pos={0 0}, upload=371802]: Done begin upload segment.
Aug 23 14:53:14.686578557 (357h27m9.117658688s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687030563 (357h27m9.118110694s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687351386 (357h27m9.118431527s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687662001 (357h27m9.118742132s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687978075 (357h27m9.119058216s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688266948 (357h27m9.119347079s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688613230 (357h27m9.119693361s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688908004 (357h27m9.119988135s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689190886 (357h27m9.120271017s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689540414 (357h27m9.120620545s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690019381 (357h27m9.121099512s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690316841 (357h27m9.121396972s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690607566 (357h27m9.121687697s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690879867 (357h27m9.121959998s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691151808 (357h27m9.122231939s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691407797 (357h27m9.122487928s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691694226 (357h27m9.122774357s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.692104832 (357h27m9.123184963s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:16.187338881 (357h27m10.618419002s) [upload=371802]: Next returned nil segment
Aug 23 14:53:16.187362156 (357h27m10.618442287s) [upload=371802]: Waiting for error group managing segments...
Aug 23 14:53:16.564984571 (357h27m10.996064712s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.656767810 (357h27m11.087847941s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.673119737 (357h27m11.104199868s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.701572207 (357h27m11.132652338s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.803065146 (357h27m11.234145277s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.833099403 (357h27m11.264179534s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.834445559 (357h27m11.26552569s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.855841568 (357h27m11.286921699s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.872230768 (357h27m11.303310899s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.962297120 (357h27m11.393377241s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.983119272 (357h27m11.414199403s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.990262750 (357h27m11.421342881s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.048189307 (357h27m11.479269448s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071859558 (357h27m11.502939689s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071888494 (357h27m11.502968625s) [seg_pos={0 0}, upload=371802]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:53:17.072009154 (357h27m11.503089285s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aGm17yvKPgD3nqcQyoMVvgM3gqXDDrKPNzXJY1xf2uBdcfobS): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072168800 (357h27m11.503248931s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf): protocol: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection
Aug 23 14:53:17.072218625 (357h27m11.503298756s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection
Aug 23 14:53:17.072310891 (357h27m11.503391022s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072352371 (357h27m11.503432492s) [seg_pos={0 0}, upload=371802]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:53:17.072435190 (357h27m11.503515321s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:53:17.072440520 (357h27m11.503520651s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:53:17.414793920 (357h27m11.845874052s) [upload=371802]: Done uploading segments...
Aug 23 14:53:17.414812586 (357h27m11.845892717s) [upload=371802]: Upload finished. err: <nil>
Duration: 6.492399489s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...............X.......................X...................X] duration:
1.726889524s
FAIL (2025-08-23 14:57:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 3a643a909e105b85e82a5a0738bde0b178866a88bf77126d042b98bfad6ed83d != before 7c5285e6e06c880fa5448453da287f2bb9bbb2fd371f54a13ff8f88b5c9be385 ======LOGS====== Aug 23 14:57:19.162870253 (357h31m13.593950384s) [upload=371871]: Starting upload Aug 23 14:57:19.162921210 (357h31m13.594001341s) [upload=371871]: Done starting upload Aug 23 14:57:19.162976887 (357h31m13.594057018s) [upload=371871]: Uploading segments... Aug 23 14:57:19.163018857 (357h31m13.594098988s) [upload=371871]: Got next segment. Inline: true Aug 23 14:57:19.163022904 (357h31m13.594103035s) [upload=371871]: Waiting for error group managing segments... Aug 23 14:57:19.163061539 (357h31m13.59414167s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:57:19.163066037 (357h31m13.594146158s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:57:19.163068772 (357h31m13.594148903s) [upload=371871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:57:19.502542644 (357h31m13.933622775s) [upload=371871]: Done uploading segments... Aug 23 14:57:19.502562963 (357h31m13.933643095s) [upload=371871]: Upload finished. err: <nil>
Duration: 1.726920664s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [................X....X.X....................X.........X.....] duration:
6.348248158s
FAIL (2025-08-23 14:57:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 78ea70faf07caf1b60b4ca1c3d20ab82be1edff82179dffb3e7c40e6fba4e870 != before fb7a1b9a15ab61c6d2d8961ed8581d81a88ea41c13daddb60db1567d3816571c
======LOGS======
Aug 23 14:53:14.233761975 (357h27m8.664842106s) [upload=371802]: Starting upload
Aug 23 14:53:14.233969873 (357h27m8.665050004s) [upload=371802]: Done starting upload
Aug 23 14:53:14.234025399 (357h27m8.66510553s) [upload=371802]: Uploading segments...
Aug 23 14:53:14.234192869 (357h27m8.665273s) [upload=371802]: Got next segment. Inline: false
Aug 23 14:53:14.234222967 (357h27m8.665303098s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:53:14.234227315 (357h27m8.665307446s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:53:14.686493825 (357h27m9.117573956s) [seg_pos={0 0}, upload=371802]: Begin upload segment...
Aug 23 14:53:14.686550944 (357h27m9.117631075s) [seg_pos={0 0}, upload=371802]: Done begin upload segment.
Aug 23 14:53:14.686578557 (357h27m9.117658688s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687030563 (357h27m9.118110694s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687351386 (357h27m9.118431527s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687662001 (357h27m9.118742132s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687978075 (357h27m9.119058216s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688266948 (357h27m9.119347079s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688613230 (357h27m9.119693361s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688908004 (357h27m9.119988135s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689190886 (357h27m9.120271017s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689540414 (357h27m9.120620545s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690019381 (357h27m9.121099512s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690316841 (357h27m9.121396972s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690607566 (357h27m9.121687697s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690879867 (357h27m9.121959998s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691151808 (357h27m9.122231939s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691407797 (357h27m9.122487928s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691694226 (357h27m9.122774357s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.692104832 (357h27m9.123184963s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:16.187338881 (357h27m10.618419002s) [upload=371802]: Next returned nil segment
Aug 23 14:53:16.187362156 (357h27m10.618442287s) [upload=371802]: Waiting for error group managing segments...
Aug 23 14:53:16.564984571 (357h27m10.996064712s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.656767810 (357h27m11.087847941s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.673119737 (357h27m11.104199868s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.701572207 (357h27m11.132652338s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.803065146 (357h27m11.234145277s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.833099403 (357h27m11.264179534s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.834445559 (357h27m11.26552569s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.855841568 (357h27m11.286921699s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.872230768 (357h27m11.303310899s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.962297120 (357h27m11.393377241s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.983119272 (357h27m11.414199403s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.990262750 (357h27m11.421342881s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.048189307 (357h27m11.479269448s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071859558 (357h27m11.502939689s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071888494 (357h27m11.502968625s) [seg_pos={0 0}, upload=371802]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:53:17.072009154 (357h27m11.503089285s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aGm17yvKPgD3nqcQyoMVvgM3gqXDDrKPNzXJY1xf2uBdcfobS): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072168800 (357h27m11.503248931s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf): protocol: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection
Aug 23 14:53:17.072218625 (357h27m11.503298756s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection
Aug 23 14:53:17.072310891 (357h27m11.503391022s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072352371 (357h27m11.503432492s) [seg_pos={0 0}, upload=371802]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:53:17.072435190 (357h27m11.503515321s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:53:17.072440520 (357h27m11.503520651s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:53:17.414793920 (357h27m11.845874052s) [upload=371802]: Done uploading segments...
Aug 23 14:53:17.414812586 (357h27m11.845892717s) [upload=371802]: Upload finished. err: <nil>
Duration: 6.348282182s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................X.......................X...................] duration:
2.042052038s
FAIL (2025-08-23 14:56:42 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.042082026s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................X.......................X..................] duration:
2.199325947s
FAIL (2025-08-23 14:56:14 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.199375753s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.................X....X.X....................X.........X....] duration:
6.292794703s
FAIL (2025-08-23 14:56:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 78ea70faf07caf1b60b4ca1c3d20ab82be1edff82179dffb3e7c40e6fba4e870 != before fb7a1b9a15ab61c6d2d8961ed8581d81a88ea41c13daddb60db1567d3816571c
======LOGS======
Aug 23 14:53:14.233761975 (357h27m8.664842106s) [upload=371802]: Starting upload
Aug 23 14:53:14.233969873 (357h27m8.665050004s) [upload=371802]: Done starting upload
Aug 23 14:53:14.234025399 (357h27m8.66510553s) [upload=371802]: Uploading segments...
Aug 23 14:53:14.234192869 (357h27m8.665273s) [upload=371802]: Got next segment. Inline: false
Aug 23 14:53:14.234222967 (357h27m8.665303098s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:53:14.234227315 (357h27m8.665307446s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:53:14.686493825 (357h27m9.117573956s) [seg_pos={0 0}, upload=371802]: Begin upload segment...
Aug 23 14:53:14.686550944 (357h27m9.117631075s) [seg_pos={0 0}, upload=371802]: Done begin upload segment.
Aug 23 14:53:14.686578557 (357h27m9.117658688s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687030563 (357h27m9.118110694s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687351386 (357h27m9.118431527s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687662001 (357h27m9.118742132s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687978075 (357h27m9.119058216s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688266948 (357h27m9.119347079s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688613230 (357h27m9.119693361s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688908004 (357h27m9.119988135s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689190886 (357h27m9.120271017s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689540414 (357h27m9.120620545s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690019381 (357h27m9.121099512s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690316841 (357h27m9.121396972s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690607566 (357h27m9.121687697s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690879867 (357h27m9.121959998s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691151808 (357h27m9.122231939s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691407797 (357h27m9.122487928s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691694226 (357h27m9.122774357s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.692104832 (357h27m9.123184963s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:16.187338881 (357h27m10.618419002s) [upload=371802]: Next returned nil segment
Aug 23 14:53:16.187362156 (357h27m10.618442287s) [upload=371802]: Waiting for error group managing segments...
Aug 23 14:53:16.564984571 (357h27m10.996064712s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.656767810 (357h27m11.087847941s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.673119737 (357h27m11.104199868s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.701572207 (357h27m11.132652338s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.803065146 (357h27m11.234145277s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.833099403 (357h27m11.264179534s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.834445559 (357h27m11.26552569s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.855841568 (357h27m11.286921699s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.872230768 (357h27m11.303310899s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.962297120 (357h27m11.393377241s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.983119272 (357h27m11.414199403s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.990262750 (357h27m11.421342881s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.048189307 (357h27m11.479269448s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071859558 (357h27m11.502939689s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071888494 (357h27m11.502968625s) [seg_pos={0 0}, upload=371802]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:53:17.072009154 (357h27m11.503089285s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aGm17yvKPgD3nqcQyoMVvgM3gqXDDrKPNzXJY1xf2uBdcfobS): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072168800 (357h27m11.503248931s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf): protocol: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection
Aug 23 14:53:17.072218625 (357h27m11.503298756s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection
Aug 23 14:53:17.072310891 (357h27m11.503391022s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072352371 (357h27m11.503432492s) [seg_pos={0 0}, upload=371802]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:53:17.072435190 (357h27m11.503515321s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:53:17.072440520 (357h27m11.503520651s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:53:17.414793920 (357h27m11.845874052s) [upload=371802]: Done uploading segments...
Aug 23 14:53:17.414812586 (357h27m11.845892717s) [upload=371802]: Upload finished. err: <nil>
Duration: 6.292823247s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..................X....X.X....................X.........X...] duration:
6.272629899s
FAIL (2025-08-23 14:55:30 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 78ea70faf07caf1b60b4ca1c3d20ab82be1edff82179dffb3e7c40e6fba4e870 != before fb7a1b9a15ab61c6d2d8961ed8581d81a88ea41c13daddb60db1567d3816571c
======LOGS======
Aug 23 14:53:14.233761975 (357h27m8.664842106s) [upload=371802]: Starting upload
Aug 23 14:53:14.233969873 (357h27m8.665050004s) [upload=371802]: Done starting upload
Aug 23 14:53:14.234025399 (357h27m8.66510553s) [upload=371802]: Uploading segments...
Aug 23 14:53:14.234192869 (357h27m8.665273s) [upload=371802]: Got next segment. Inline: false
Aug 23 14:53:14.234222967 (357h27m8.665303098s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:53:14.234227315 (357h27m8.665307446s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:53:14.686493825 (357h27m9.117573956s) [seg_pos={0 0}, upload=371802]: Begin upload segment...
Aug 23 14:53:14.686550944 (357h27m9.117631075s) [seg_pos={0 0}, upload=371802]: Done begin upload segment.
Aug 23 14:53:14.686578557 (357h27m9.117658688s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687030563 (357h27m9.118110694s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687351386 (357h27m9.118431527s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687662001 (357h27m9.118742132s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687978075 (357h27m9.119058216s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688266948 (357h27m9.119347079s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688613230 (357h27m9.119693361s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688908004 (357h27m9.119988135s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689190886 (357h27m9.120271017s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689540414 (357h27m9.120620545s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690019381 (357h27m9.121099512s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690316841 (357h27m9.121396972s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690607566 (357h27m9.121687697s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690879867 (357h27m9.121959998s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691151808 (357h27m9.122231939s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691407797 (357h27m9.122487928s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691694226 (357h27m9.122774357s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.692104832 (357h27m9.123184963s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:16.187338881 (357h27m10.618419002s) [upload=371802]: Next returned nil segment
Aug 23 14:53:16.187362156 (357h27m10.618442287s) [upload=371802]: Waiting for error group managing segments...
Aug 23 14:53:16.564984571 (357h27m10.996064712s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.656767810 (357h27m11.087847941s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.673119737 (357h27m11.104199868s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.701572207 (357h27m11.132652338s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.803065146 (357h27m11.234145277s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.833099403 (357h27m11.264179534s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.834445559 (357h27m11.26552569s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.855841568 (357h27m11.286921699s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.872230768 (357h27m11.303310899s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.962297120 (357h27m11.393377241s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.983119272 (357h27m11.414199403s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.990262750 (357h27m11.421342881s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.048189307 (357h27m11.479269448s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071859558 (357h27m11.502939689s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071888494 (357h27m11.502968625s) [seg_pos={0 0}, upload=371802]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:53:17.072009154 (357h27m11.503089285s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aGm17yvKPgD3nqcQyoMVvgM3gqXDDrKPNzXJY1xf2uBdcfobS): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072168800 (357h27m11.503248931s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf): protocol: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection
Aug 23 14:53:17.072218625 (357h27m11.503298756s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection
Aug 23 14:53:17.072310891 (357h27m11.503391022s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072352371 (357h27m11.503432492s) [seg_pos={0 0}, upload=371802]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:53:17.072435190 (357h27m11.503515321s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:53:17.072440520 (357h27m11.503520651s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:53:17.414793920 (357h27m11.845874052s) [upload=371802]: Done uploading segments...
Aug 23 14:53:17.414812586 (357h27m11.845892717s) [upload=371802]: Upload finished. err: <nil>
Duration: 6.272652322s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................X.......................X.................] duration:
1.86906828s
FAIL (2025-08-23 14:55:11 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.869099029s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [X..................X....X.X....................X.........X..] duration:
6.17518046s
FAIL (2025-08-23 14:54:53 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 78ea70faf07caf1b60b4ca1c3d20ab82be1edff82179dffb3e7c40e6fba4e870 != before fb7a1b9a15ab61c6d2d8961ed8581d81a88ea41c13daddb60db1567d3816571c
======LOGS======
Aug 23 14:53:14.233761975 (357h27m8.664842106s) [upload=371802]: Starting upload
Aug 23 14:53:14.233969873 (357h27m8.665050004s) [upload=371802]: Done starting upload
Aug 23 14:53:14.234025399 (357h27m8.66510553s) [upload=371802]: Uploading segments...
Aug 23 14:53:14.234192869 (357h27m8.665273s) [upload=371802]: Got next segment. Inline: false
Aug 23 14:53:14.234222967 (357h27m8.665303098s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:53:14.234227315 (357h27m8.665307446s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:53:14.686493825 (357h27m9.117573956s) [seg_pos={0 0}, upload=371802]: Begin upload segment...
Aug 23 14:53:14.686550944 (357h27m9.117631075s) [seg_pos={0 0}, upload=371802]: Done begin upload segment.
Aug 23 14:53:14.686578557 (357h27m9.117658688s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687030563 (357h27m9.118110694s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687351386 (357h27m9.118431527s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687662001 (357h27m9.118742132s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687978075 (357h27m9.119058216s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688266948 (357h27m9.119347079s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688613230 (357h27m9.119693361s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688908004 (357h27m9.119988135s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689190886 (357h27m9.120271017s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689540414 (357h27m9.120620545s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690019381 (357h27m9.121099512s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690316841 (357h27m9.121396972s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690607566 (357h27m9.121687697s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690879867 (357h27m9.121959998s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691151808 (357h27m9.122231939s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691407797 (357h27m9.122487928s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691694226 (357h27m9.122774357s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.692104832 (357h27m9.123184963s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:16.187338881 (357h27m10.618419002s) [upload=371802]: Next returned nil segment
Aug 23 14:53:16.187362156 (357h27m10.618442287s) [upload=371802]: Waiting for error group managing segments...
Aug 23 14:53:16.564984571 (357h27m10.996064712s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.656767810 (357h27m11.087847941s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.673119737 (357h27m11.104199868s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.701572207 (357h27m11.132652338s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.803065146 (357h27m11.234145277s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.833099403 (357h27m11.264179534s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.834445559 (357h27m11.26552569s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.855841568 (357h27m11.286921699s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.872230768 (357h27m11.303310899s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.962297120 (357h27m11.393377241s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.983119272 (357h27m11.414199403s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.990262750 (357h27m11.421342881s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.048189307 (357h27m11.479269448s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071859558 (357h27m11.502939689s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071888494 (357h27m11.502968625s) [seg_pos={0 0}, upload=371802]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:53:17.072009154 (357h27m11.503089285s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aGm17yvKPgD3nqcQyoMVvgM3gqXDDrKPNzXJY1xf2uBdcfobS): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072168800 (357h27m11.503248931s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf): protocol: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection
Aug 23 14:53:17.072218625 (357h27m11.503298756s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection
Aug 23 14:53:17.072310891 (357h27m11.503391022s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072352371 (357h27m11.503432492s) [seg_pos={0 0}, upload=371802]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:53:17.072435190 (357h27m11.503515321s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:53:17.072440520 (357h27m11.503520651s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:53:17.414793920 (357h27m11.845874052s) [upload=371802]: Done uploading segments...
Aug 23 14:53:17.414812586 (357h27m11.845892717s) [upload=371802]: Upload finished. err: <nil>
Duration: 6.175204385s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.X..................X....X.X....................X.........X.] duration:
6.47924637s
FAIL (2025-08-23 14:54:25 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 78ea70faf07caf1b60b4ca1c3d20ab82be1edff82179dffb3e7c40e6fba4e870 != before fb7a1b9a15ab61c6d2d8961ed8581d81a88ea41c13daddb60db1567d3816571c
======LOGS======
Aug 23 14:53:14.233761975 (357h27m8.664842106s) [upload=371802]: Starting upload
Aug 23 14:53:14.233969873 (357h27m8.665050004s) [upload=371802]: Done starting upload
Aug 23 14:53:14.234025399 (357h27m8.66510553s) [upload=371802]: Uploading segments...
Aug 23 14:53:14.234192869 (357h27m8.665273s) [upload=371802]: Got next segment. Inline: false
Aug 23 14:53:14.234222967 (357h27m8.665303098s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:53:14.234227315 (357h27m8.665307446s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:53:14.686493825 (357h27m9.117573956s) [seg_pos={0 0}, upload=371802]: Begin upload segment...
Aug 23 14:53:14.686550944 (357h27m9.117631075s) [seg_pos={0 0}, upload=371802]: Done begin upload segment.
Aug 23 14:53:14.686578557 (357h27m9.117658688s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687030563 (357h27m9.118110694s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687351386 (357h27m9.118431527s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687662001 (357h27m9.118742132s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687978075 (357h27m9.119058216s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688266948 (357h27m9.119347079s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688613230 (357h27m9.119693361s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688908004 (357h27m9.119988135s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689190886 (357h27m9.120271017s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689540414 (357h27m9.120620545s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690019381 (357h27m9.121099512s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690316841 (357h27m9.121396972s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690607566 (357h27m9.121687697s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690879867 (357h27m9.121959998s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691151808 (357h27m9.122231939s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691407797 (357h27m9.122487928s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691694226 (357h27m9.122774357s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.692104832 (357h27m9.123184963s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:16.187338881 (357h27m10.618419002s) [upload=371802]: Next returned nil segment
Aug 23 14:53:16.187362156 (357h27m10.618442287s) [upload=371802]: Waiting for error group managing segments...
Aug 23 14:53:16.564984571 (357h27m10.996064712s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.656767810 (357h27m11.087847941s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.673119737 (357h27m11.104199868s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.701572207 (357h27m11.132652338s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.803065146 (357h27m11.234145277s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.833099403 (357h27m11.264179534s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.834445559 (357h27m11.26552569s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.855841568 (357h27m11.286921699s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.872230768 (357h27m11.303310899s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.962297120 (357h27m11.393377241s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.983119272 (357h27m11.414199403s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.990262750 (357h27m11.421342881s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.048189307 (357h27m11.479269448s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071859558 (357h27m11.502939689s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071888494 (357h27m11.502968625s) [seg_pos={0 0}, upload=371802]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:53:17.072009154 (357h27m11.503089285s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aGm17yvKPgD3nqcQyoMVvgM3gqXDDrKPNzXJY1xf2uBdcfobS): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072168800 (357h27m11.503248931s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf): protocol: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection
Aug 23 14:53:17.072218625 (357h27m11.503298756s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection
Aug 23 14:53:17.072310891 (357h27m11.503391022s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072352371 (357h27m11.503432492s) [seg_pos={0 0}, upload=371802]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:53:17.072435190 (357h27m11.503515321s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:53:17.072440520 (357h27m11.503520651s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:53:17.414793920 (357h27m11.845874052s) [upload=371802]: Done uploading segments...
Aug 23 14:53:17.414812586 (357h27m11.845892717s) [upload=371802]: Upload finished. err: <nil>
Duration: 6.479272602s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...................X.......................X................] duration:
2.015296373s
FAIL (2025-08-23 14:54:17 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.015340188s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....................X.......................X...............] duration:
2.046021169s
FAIL (2025-08-23 14:53:40 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.046047731s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [..X..................X....X.X....................X.........X] duration:
6.658522786s
FAIL (2025-08-23 14:53:20 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 78ea70faf07caf1b60b4ca1c3d20ab82be1edff82179dffb3e7c40e6fba4e870 != before fb7a1b9a15ab61c6d2d8961ed8581d81a88ea41c13daddb60db1567d3816571c
======LOGS======
Aug 23 14:53:14.233761975 (357h27m8.664842106s) [upload=371802]: Starting upload
Aug 23 14:53:14.233969873 (357h27m8.665050004s) [upload=371802]: Done starting upload
Aug 23 14:53:14.234025399 (357h27m8.66510553s) [upload=371802]: Uploading segments...
Aug 23 14:53:14.234192869 (357h27m8.665273s) [upload=371802]: Got next segment. Inline: false
Aug 23 14:53:14.234222967 (357h27m8.665303098s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:53:14.234227315 (357h27m8.665307446s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:53:14.686493825 (357h27m9.117573956s) [seg_pos={0 0}, upload=371802]: Begin upload segment...
Aug 23 14:53:14.686550944 (357h27m9.117631075s) [seg_pos={0 0}, upload=371802]: Done begin upload segment.
Aug 23 14:53:14.686578557 (357h27m9.117658688s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687030563 (357h27m9.118110694s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687351386 (357h27m9.118431527s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687662001 (357h27m9.118742132s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.687978075 (357h27m9.119058216s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688266948 (357h27m9.119347079s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688613230 (357h27m9.119693361s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.688908004 (357h27m9.119988135s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689190886 (357h27m9.120271017s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.689540414 (357h27m9.120620545s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690019381 (357h27m9.121099512s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690316841 (357h27m9.121396972s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690607566 (357h27m9.121687697s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.690879867 (357h27m9.121959998s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691151808 (357h27m9.122231939s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691407797 (357h27m9.122487928s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.691694226 (357h27m9.122774357s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:14.692104832 (357h27m9.123184963s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Uploading piece...
Aug 23 14:53:16.187338881 (357h27m10.618419002s) [upload=371802]: Next returned nil segment
Aug 23 14:53:16.187362156 (357h27m10.618442287s) [upload=371802]: Waiting for error group managing segments...
Aug 23 14:53:16.564984571 (357h27m10.996064712s) [noise=true , address=79.127.219.47:4068 , piece_id=OZ4YKYMWYEJYAG6AUU6OEQO6TDIO5KNFF5C5U24DSYMS3S74T4GA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.656767810 (357h27m11.087847941s) [noise=true , address=79.127.201.215:4357 , piece_id=TV5RGON4KFZM6SKQ4QYCPHF6UCCCT7RZPZTOIDG2Z2AVBHSEEW6A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.673119737 (357h27m11.104199868s) [noise=true , address=104.36.82.18:4377 , piece_id=HF7P3MOVH5H3HQUNYFHMB7A2LJFL7TQYIEQV72OHXHXWO37QGXWA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.701572207 (357h27m11.132652338s) [noise=true , address=104.36.82.23:4476 , piece_id=RG4XT4XQS33CQQMK43WH6K3QMU7C2TAMVVKEDGWETVNMUWN6F4LQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.803065146 (357h27m11.234145277s) [noise=true , address=104.36.82.50:4039 , piece_id=UYB3O2VDOY5Q34L3MMEHK3F7WC5OSXBYH6G6RT23MJSSH6Z5IT2A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.833099403 (357h27m11.264179534s) [noise=true , address=77.90.31.10:4530 , piece_id=AR2NFEVWB3G5XVRMRPLE575EZIYFQN5XOUGKWUYMQAQRTXVBQDQQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.834445559 (357h27m11.26552569s) [noise=true , address=77.90.31.19:4848 , piece_id=TBP2GGI5ZT6DCWQKMMBS6ANL5KTF6Q5X7LRAHB7UB5OBECQREOFQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.855841568 (357h27m11.286921699s) [noise=true , address=77.90.31.13:4651 , piece_id=B4FOOTNBYBMSIGHYFXKKSET24JTTUA26A27L2CTT7QFQSSHEU5EA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.872230768 (357h27m11.303310899s) [noise=true , address=45.91.116.10:4090 , piece_id=KYRJQ2H2XIKO7YOPHLZIGT5UJQJ2BWBO7J6JB5FOJOM4UQJPG34A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.962297120 (357h27m11.393377241s) [noise=true , address=77.90.31.11:4565 , piece_id=4DUVXT7KSVUWTODCT4CPFK4NVYWX47AQCSUVNQOTZXV4QUJJEMUA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.983119272 (357h27m11.414199403s) [noise=true , address=45.91.116.4:4947 , piece_id=D2CBJN5ES6E35AHGU63EWCSQY622LML77GTGKK5NVH4OVMB7M74Q, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:16.990262750 (357h27m11.421342881s) [noise=true , address=45.91.116.11:4126 , piece_id=YJH4U3A3VYQOVV7INW7ZL35KIS7LWPT6OMQV4G62UTJ5RVNDML7A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.048189307 (357h27m11.479269448s) [noise=true , address=104.36.82.45:4919 , piece_id=IUI6JVQ5ZH6ZP7OFNP7UT3NSLZV5IWGAK6TEC3ATXBO7X6XSTYJA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071859558 (357h27m11.502939689s) [noise=true , address=45.91.116.8:4060 , piece_id=VYZ37EVKSOKHYKO4XRUQIJZU2XEUDFICP2S6KRVJHFWMFJX5IKUQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: <nil>
Aug 23 14:53:17.071888494 (357h27m11.502968625s) [seg_pos={0 0}, upload=371802]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:53:17.072009154 (357h27m11.503089285s) [noise=true , address=104.36.82.20:4465 , piece_id=VB3MBL7UFC3NPDP7HNNPMB3P4OYSFM7PZLGPBOTY4NHT6MEJM63A, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aGm17yvKPgD3nqcQyoMVvgM3gqXDDrKPNzXJY1xf2uBdcfobS): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072168800 (357h27m11.503248931s) [noise=true , address=104.36.82.60:4343 , piece_id=U32JFO3AYH6VVLM3UYW7Q26SKZF5KENRC6BPVQLUCNUE3NMPGJZA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf): protocol: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43510->104.36.82.60:4343: use of closed network connection
Aug 23 14:53:17.072218625 (357h27m11.503298756s) [noise=true , address=104.36.82.5:4862 , piece_id=VLFF5ULHXADARYU7BGORS5E3F6VLL5B3CJ64XWJWBS5G7NLR6ZEQ, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38442->104.36.82.5:4862: use of closed network connection
Aug 23 14:53:17.072310891 (357h27m11.503391022s) [noise=true , address=104.36.82.44:4901 , piece_id=EFRO6CYDDCVJJWFMPUWS634L7CQHWF7HCWGFLMJT4PGUJ2BB3JKA, seg_pos={0 0}, upload=371802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1): protocol: expected piece hash; context canceled; EOF
Aug 23 14:53:17.072352371 (357h27m11.503432492s) [seg_pos={0 0}, upload=371802]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:53:17.072435190 (357h27m11.503515321s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:53:17.072440520 (357h27m11.503520651s) [upload=371802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:53:17.414793920 (357h27m11.845874052s) [upload=371802]: Done uploading segments...
Aug 23 14:53:17.414812586 (357h27m11.845892717s) [upload=371802]: Upload finished. err: <nil>
Duration: 6.658553925s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....................X.......................X..............] duration:
1.829169644s
FAIL (2025-08-23 14:52:49 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.829252963s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...X..................X....X.X....................X.........] duration:
6.647405697s
FAIL (2025-08-23 14:52:06 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:45:24.054384699 (357h19m18.48546484s) [upload=371666]: Starting upload
Aug 23 14:45:24.054471174 (357h19m18.485551305s) [upload=371666]: Done starting upload
Aug 23 14:45:24.054563761 (357h19m18.485643892s) [upload=371666]: Uploading segments...
Aug 23 14:45:24.054700883 (357h19m18.485781014s) [upload=371666]: Got next segment. Inline: false
Aug 23 14:45:24.054758313 (357h19m18.485838484s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:45:24.054767090 (357h19m18.485847261s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:45:24.512644204 (357h19m18.943724336s) [seg_pos={0 0}, upload=371666]: Begin upload segment...
Aug 23 14:45:24.512685454 (357h19m18.943765585s) [seg_pos={0 0}, upload=371666]: Done begin upload segment.
Aug 23 14:45:24.512705382 (357h19m18.943785513s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.512739748 (357h19m18.943819879s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513073596 (357h19m18.944153727s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513189728 (357h19m18.944269859s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513374562 (357h19m18.944454693s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513700065 (357h19m18.944780206s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514065845 (357h19m18.945145976s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514450741 (357h19m18.945530872s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514690520 (357h19m18.945770651s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514915771 (357h19m18.945995902s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515141302 (357h19m18.946221433s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515335314 (357h19m18.946415446s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515543913 (357h19m18.946624044s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515757332 (357h19m18.946837463s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515967714 (357h19m18.947047835s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516176834 (357h19m18.947256965s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516377859 (357h19m18.94745798s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516614882 (357h19m18.947695013s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:26.051354502 (357h19m20.482434633s) [upload=371666]: Next returned nil segment
Aug 23 14:45:26.051372857 (357h19m20.482452988s) [upload=371666]: Waiting for error group managing segments...
Aug 23 14:45:26.499241712 (357h19m20.930321843s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.532419114 (357h19m20.963499245s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.628848182 (357h19m21.059928313s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.666022381 (357h19m21.097102522s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.668507786 (357h19m21.099587917s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.717353037 (357h19m21.148433168s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.748823854 (357h19m21.179903995s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.763937462 (357h19m21.195017593s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.769131070 (357h19m21.200211201s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.785056963 (357h19m21.216137094s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.786918465 (357h19m21.217998596s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.853343923 (357h19m21.284424054s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.879991480 (357h19m21.311071611s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889610165 (357h19m21.320690296s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889676141 (357h19m21.320756272s) [seg_pos={0 0}, upload=371666]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:45:26.890072289 (357h19m21.32115242s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890132134 (357h19m21.321212265s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890158935 (357h19m21.321239066s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE): protocol: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection
Aug 23 14:45:26.890228298 (357h19m21.321308419s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct): protocol: rpc: dial tcp 104.36.82.23:4472: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 23 14:45:26.890253176 (357h19m21.321333317s) [seg_pos={0 0}, upload=371666]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:45:26.890333268 (357h19m21.321413399s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:45:26.890338609 (357h19m21.32141874s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:45:27.223463992 (357h19m21.654544124s) [upload=371666]: Done uploading segments...
Aug 23 14:45:27.223483129 (357h19m21.65456326s) [upload=371666]: Upload finished. err: <nil>
Duration: 6.647440363s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [....X..................X....X.X....................X........] duration:
6.704088662s
FAIL (2025-08-23 14:51:39 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:45:24.054384699 (357h19m18.48546484s) [upload=371666]: Starting upload
Aug 23 14:45:24.054471174 (357h19m18.485551305s) [upload=371666]: Done starting upload
Aug 23 14:45:24.054563761 (357h19m18.485643892s) [upload=371666]: Uploading segments...
Aug 23 14:45:24.054700883 (357h19m18.485781014s) [upload=371666]: Got next segment. Inline: false
Aug 23 14:45:24.054758313 (357h19m18.485838484s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:45:24.054767090 (357h19m18.485847261s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:45:24.512644204 (357h19m18.943724336s) [seg_pos={0 0}, upload=371666]: Begin upload segment...
Aug 23 14:45:24.512685454 (357h19m18.943765585s) [seg_pos={0 0}, upload=371666]: Done begin upload segment.
Aug 23 14:45:24.512705382 (357h19m18.943785513s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.512739748 (357h19m18.943819879s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513073596 (357h19m18.944153727s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513189728 (357h19m18.944269859s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513374562 (357h19m18.944454693s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513700065 (357h19m18.944780206s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514065845 (357h19m18.945145976s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514450741 (357h19m18.945530872s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514690520 (357h19m18.945770651s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514915771 (357h19m18.945995902s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515141302 (357h19m18.946221433s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515335314 (357h19m18.946415446s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515543913 (357h19m18.946624044s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515757332 (357h19m18.946837463s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515967714 (357h19m18.947047835s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516176834 (357h19m18.947256965s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516377859 (357h19m18.94745798s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516614882 (357h19m18.947695013s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:26.051354502 (357h19m20.482434633s) [upload=371666]: Next returned nil segment
Aug 23 14:45:26.051372857 (357h19m20.482452988s) [upload=371666]: Waiting for error group managing segments...
Aug 23 14:45:26.499241712 (357h19m20.930321843s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.532419114 (357h19m20.963499245s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.628848182 (357h19m21.059928313s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.666022381 (357h19m21.097102522s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.668507786 (357h19m21.099587917s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.717353037 (357h19m21.148433168s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.748823854 (357h19m21.179903995s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.763937462 (357h19m21.195017593s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.769131070 (357h19m21.200211201s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.785056963 (357h19m21.216137094s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.786918465 (357h19m21.217998596s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.853343923 (357h19m21.284424054s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.879991480 (357h19m21.311071611s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889610165 (357h19m21.320690296s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889676141 (357h19m21.320756272s) [seg_pos={0 0}, upload=371666]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:45:26.890072289 (357h19m21.32115242s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890132134 (357h19m21.321212265s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890158935 (357h19m21.321239066s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE): protocol: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection
Aug 23 14:45:26.890228298 (357h19m21.321308419s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct): protocol: rpc: dial tcp 104.36.82.23:4472: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 23 14:45:26.890253176 (357h19m21.321333317s) [seg_pos={0 0}, upload=371666]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:45:26.890333268 (357h19m21.321413399s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:45:26.890338609 (357h19m21.32141874s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:45:27.223463992 (357h19m21.654544124s) [upload=371666]: Done uploading segments...
Aug 23 14:45:27.223483129 (357h19m21.65456326s) [upload=371666]: Upload finished. err: <nil>
Duration: 6.704109903s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......................X.......................X.............] duration:
2.150165039s
FAIL (2025-08-23 14:51:37 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.150193175s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......................X.......................X............] duration:
1.99026274s
FAIL (2025-08-23 14:51:03 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.990295043s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.....X..................X....X.X....................X.......] duration:
6.795005441s
FAIL (2025-08-23 14:50:43 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:45:24.054384699 (357h19m18.48546484s) [upload=371666]: Starting upload
Aug 23 14:45:24.054471174 (357h19m18.485551305s) [upload=371666]: Done starting upload
Aug 23 14:45:24.054563761 (357h19m18.485643892s) [upload=371666]: Uploading segments...
Aug 23 14:45:24.054700883 (357h19m18.485781014s) [upload=371666]: Got next segment. Inline: false
Aug 23 14:45:24.054758313 (357h19m18.485838484s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:45:24.054767090 (357h19m18.485847261s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:45:24.512644204 (357h19m18.943724336s) [seg_pos={0 0}, upload=371666]: Begin upload segment...
Aug 23 14:45:24.512685454 (357h19m18.943765585s) [seg_pos={0 0}, upload=371666]: Done begin upload segment.
Aug 23 14:45:24.512705382 (357h19m18.943785513s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.512739748 (357h19m18.943819879s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513073596 (357h19m18.944153727s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513189728 (357h19m18.944269859s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513374562 (357h19m18.944454693s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513700065 (357h19m18.944780206s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514065845 (357h19m18.945145976s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514450741 (357h19m18.945530872s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514690520 (357h19m18.945770651s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514915771 (357h19m18.945995902s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515141302 (357h19m18.946221433s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515335314 (357h19m18.946415446s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515543913 (357h19m18.946624044s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515757332 (357h19m18.946837463s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515967714 (357h19m18.947047835s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516176834 (357h19m18.947256965s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516377859 (357h19m18.94745798s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516614882 (357h19m18.947695013s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:26.051354502 (357h19m20.482434633s) [upload=371666]: Next returned nil segment
Aug 23 14:45:26.051372857 (357h19m20.482452988s) [upload=371666]: Waiting for error group managing segments...
Aug 23 14:45:26.499241712 (357h19m20.930321843s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.532419114 (357h19m20.963499245s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.628848182 (357h19m21.059928313s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.666022381 (357h19m21.097102522s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.668507786 (357h19m21.099587917s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.717353037 (357h19m21.148433168s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.748823854 (357h19m21.179903995s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.763937462 (357h19m21.195017593s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.769131070 (357h19m21.200211201s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.785056963 (357h19m21.216137094s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.786918465 (357h19m21.217998596s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.853343923 (357h19m21.284424054s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.879991480 (357h19m21.311071611s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889610165 (357h19m21.320690296s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889676141 (357h19m21.320756272s) [seg_pos={0 0}, upload=371666]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:45:26.890072289 (357h19m21.32115242s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890132134 (357h19m21.321212265s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890158935 (357h19m21.321239066s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE): protocol: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection
Aug 23 14:45:26.890228298 (357h19m21.321308419s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct): protocol: rpc: dial tcp 104.36.82.23:4472: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 23 14:45:26.890253176 (357h19m21.321333317s) [seg_pos={0 0}, upload=371666]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:45:26.890333268 (357h19m21.321413399s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:45:26.890338609 (357h19m21.32141874s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:45:27.223463992 (357h19m21.654544124s) [upload=371666]: Done uploading segments...
Aug 23 14:45:27.223483129 (357h19m21.65456326s) [upload=371666]: Upload finished. err: <nil>
Duration: 6.795024638s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [......X..................X....X.X....................X......] duration:
6.38266126s
FAIL (2025-08-23 14:49:56 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:45:24.054384699 (357h19m18.48546484s) [upload=371666]: Starting upload
Aug 23 14:45:24.054471174 (357h19m18.485551305s) [upload=371666]: Done starting upload
Aug 23 14:45:24.054563761 (357h19m18.485643892s) [upload=371666]: Uploading segments...
Aug 23 14:45:24.054700883 (357h19m18.485781014s) [upload=371666]: Got next segment. Inline: false
Aug 23 14:45:24.054758313 (357h19m18.485838484s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:45:24.054767090 (357h19m18.485847261s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:45:24.512644204 (357h19m18.943724336s) [seg_pos={0 0}, upload=371666]: Begin upload segment...
Aug 23 14:45:24.512685454 (357h19m18.943765585s) [seg_pos={0 0}, upload=371666]: Done begin upload segment.
Aug 23 14:45:24.512705382 (357h19m18.943785513s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.512739748 (357h19m18.943819879s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513073596 (357h19m18.944153727s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513189728 (357h19m18.944269859s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513374562 (357h19m18.944454693s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513700065 (357h19m18.944780206s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514065845 (357h19m18.945145976s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514450741 (357h19m18.945530872s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514690520 (357h19m18.945770651s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514915771 (357h19m18.945995902s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515141302 (357h19m18.946221433s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515335314 (357h19m18.946415446s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515543913 (357h19m18.946624044s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515757332 (357h19m18.946837463s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515967714 (357h19m18.947047835s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516176834 (357h19m18.947256965s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516377859 (357h19m18.94745798s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516614882 (357h19m18.947695013s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:26.051354502 (357h19m20.482434633s) [upload=371666]: Next returned nil segment
Aug 23 14:45:26.051372857 (357h19m20.482452988s) [upload=371666]: Waiting for error group managing segments...
Aug 23 14:45:26.499241712 (357h19m20.930321843s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.532419114 (357h19m20.963499245s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.628848182 (357h19m21.059928313s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.666022381 (357h19m21.097102522s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.668507786 (357h19m21.099587917s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.717353037 (357h19m21.148433168s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.748823854 (357h19m21.179903995s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.763937462 (357h19m21.195017593s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.769131070 (357h19m21.200211201s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.785056963 (357h19m21.216137094s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.786918465 (357h19m21.217998596s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.853343923 (357h19m21.284424054s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.879991480 (357h19m21.311071611s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889610165 (357h19m21.320690296s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889676141 (357h19m21.320756272s) [seg_pos={0 0}, upload=371666]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:45:26.890072289 (357h19m21.32115242s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890132134 (357h19m21.321212265s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890158935 (357h19m21.321239066s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE): protocol: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection
Aug 23 14:45:26.890228298 (357h19m21.321308419s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct): protocol: rpc: dial tcp 104.36.82.23:4472: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 23 14:45:26.890253176 (357h19m21.321333317s) [seg_pos={0 0}, upload=371666]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:45:26.890333268 (357h19m21.321413399s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:45:26.890338609 (357h19m21.32141874s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:45:27.223463992 (357h19m21.654544124s) [upload=371666]: Done uploading segments...
Aug 23 14:45:27.223483129 (357h19m21.65456326s) [upload=371666]: Upload finished. err: <nil>
Duration: 6.382689273s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........................X.......................X...........] duration:
1.919442593s
FAIL (2025-08-23 14:49:49 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.919470707s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
6.71281602s
FAIL (2025-08-23 14:49:28 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Aug 23 14:06:34.087047198 (356h40m28.518127319s) [upload=371002]: Starting upload
Aug 23 14:06:34.087103405 (356h40m28.518183536s) [upload=371002]: Done starting upload
Aug 23 14:06:34.087217894 (356h40m28.518298075s) [upload=371002]: Uploading segments...
Aug 23 14:06:34.087280084 (356h40m28.518360225s) [upload=371002]: Got next segment. Inline: false
Aug 23 14:06:34.087333556 (356h40m28.518413697s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:06:34.087345218 (356h40m28.518425379s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:06:34.329714999 (356h40m28.76079513s) [seg_pos={0 0}, upload=371002]: Begin upload segment...
Aug 23 14:06:34.329766266 (356h40m28.760846387s) [seg_pos={0 0}, upload=371002]: Done begin upload segment.
Aug 23 14:06:34.329787617 (356h40m28.760867758s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.329825380 (356h40m28.760905521s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330128550 (356h40m28.761208681s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330176731 (356h40m28.761256862s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330343832 (356h40m28.761423953s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330470363 (356h40m28.761550494s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330760569 (356h40m28.76184071s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331052808 (356h40m28.762132939s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331345137 (356h40m28.762425268s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331803835 (356h40m28.762883966s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332093870 (356h40m28.763174001s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332431015 (356h40m28.763511146s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332721310 (356h40m28.763801441s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333000254 (356h40m28.764080385s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333290029 (356h40m28.76437016s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333579212 (356h40m28.764659343s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:36.088000547 (356h40m30.519080668s) [upload=371002]: Next returned nil segment
Aug 23 14:06:36.088019162 (356h40m30.519099293s) [upload=371002]: Waiting for error group managing segments...
Aug 23 14:06:36.365114618 (356h40m30.79619475s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.384930313 (356h40m30.816010444s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.441019685 (356h40m30.872099816s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.503003134 (356h40m30.934083265s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.773706448 (356h40m31.204786629s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.794301936 (356h40m31.225382068s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.797422086 (356h40m31.228502217s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.891493522 (356h40m31.322573653s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.893130365 (356h40m31.324210496s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.926291877 (356h40m31.357372008s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.980332537 (356h40m31.411412668s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.986328430 (356h40m31.417408551s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126158031 (356h40m31.557238152s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126185253 (356h40m31.557265384s) [seg_pos={0 0}, upload=371002]: Segment reached optimal threshold of 13 pieces.
Aug 23 14:06:37.126290142 (356h40m31.557370274s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126445190 (356h40m31.557525321s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection
Aug 23 14:06:37.126492731 (356h40m31.557572862s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126560811 (356h40m31.557640952s) [seg_pos={0 0}, upload=371002]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 23 14:06:37.126686101 (356h40m31.557766232s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:06:37.126693204 (356h40m31.557773365s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:06:37.513830133 (356h40m31.944910264s) [upload=371002]: Done uploading segments...
Aug 23 14:06:37.513848849 (356h40m31.944928981s) [upload=371002]: Upload finished. err: <nil>
Duration: 6.712845507s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........................X.......................X..........] duration:
1.965334593s
FAIL (2025-08-23 14:49:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.965361163s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......X..................X....X.X....................X.....] duration:
6.88013946s
FAIL (2025-08-23 14:49:16 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:45:24.054384699 (357h19m18.48546484s) [upload=371666]: Starting upload
Aug 23 14:45:24.054471174 (357h19m18.485551305s) [upload=371666]: Done starting upload
Aug 23 14:45:24.054563761 (357h19m18.485643892s) [upload=371666]: Uploading segments...
Aug 23 14:45:24.054700883 (357h19m18.485781014s) [upload=371666]: Got next segment. Inline: false
Aug 23 14:45:24.054758313 (357h19m18.485838484s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:45:24.054767090 (357h19m18.485847261s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:45:24.512644204 (357h19m18.943724336s) [seg_pos={0 0}, upload=371666]: Begin upload segment...
Aug 23 14:45:24.512685454 (357h19m18.943765585s) [seg_pos={0 0}, upload=371666]: Done begin upload segment.
Aug 23 14:45:24.512705382 (357h19m18.943785513s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.512739748 (357h19m18.943819879s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513073596 (357h19m18.944153727s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513189728 (357h19m18.944269859s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513374562 (357h19m18.944454693s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513700065 (357h19m18.944780206s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514065845 (357h19m18.945145976s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514450741 (357h19m18.945530872s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514690520 (357h19m18.945770651s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514915771 (357h19m18.945995902s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515141302 (357h19m18.946221433s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515335314 (357h19m18.946415446s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515543913 (357h19m18.946624044s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515757332 (357h19m18.946837463s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515967714 (357h19m18.947047835s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516176834 (357h19m18.947256965s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516377859 (357h19m18.94745798s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516614882 (357h19m18.947695013s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:26.051354502 (357h19m20.482434633s) [upload=371666]: Next returned nil segment
Aug 23 14:45:26.051372857 (357h19m20.482452988s) [upload=371666]: Waiting for error group managing segments...
Aug 23 14:45:26.499241712 (357h19m20.930321843s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.532419114 (357h19m20.963499245s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.628848182 (357h19m21.059928313s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.666022381 (357h19m21.097102522s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.668507786 (357h19m21.099587917s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.717353037 (357h19m21.148433168s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.748823854 (357h19m21.179903995s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.763937462 (357h19m21.195017593s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.769131070 (357h19m21.200211201s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.785056963 (357h19m21.216137094s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.786918465 (357h19m21.217998596s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.853343923 (357h19m21.284424054s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.879991480 (357h19m21.311071611s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889610165 (357h19m21.320690296s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889676141 (357h19m21.320756272s) [seg_pos={0 0}, upload=371666]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:45:26.890072289 (357h19m21.32115242s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890132134 (357h19m21.321212265s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890158935 (357h19m21.321239066s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE): protocol: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection
Aug 23 14:45:26.890228298 (357h19m21.321308419s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct): protocol: rpc: dial tcp 104.36.82.23:4472: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 23 14:45:26.890253176 (357h19m21.321333317s) [seg_pos={0 0}, upload=371666]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:45:26.890333268 (357h19m21.321413399s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:45:26.890338609 (357h19m21.32141874s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:45:27.223463992 (357h19m21.654544124s) [upload=371666]: Done uploading segments...
Aug 23 14:45:27.223483129 (357h19m21.65456326s) [upload=371666]: Upload finished. err: <nil>
Duration: 6.880169576s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........................X.......................X.........] duration:
2.799139309s
FAIL (2025-08-23 14:48:44 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.799168113s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [........X..................X....X.X....................X....] duration:
6.614667417s
FAIL (2025-08-23 14:48:29 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:45:24.054384699 (357h19m18.48546484s) [upload=371666]: Starting upload
Aug 23 14:45:24.054471174 (357h19m18.485551305s) [upload=371666]: Done starting upload
Aug 23 14:45:24.054563761 (357h19m18.485643892s) [upload=371666]: Uploading segments...
Aug 23 14:45:24.054700883 (357h19m18.485781014s) [upload=371666]: Got next segment. Inline: false
Aug 23 14:45:24.054758313 (357h19m18.485838484s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:45:24.054767090 (357h19m18.485847261s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:45:24.512644204 (357h19m18.943724336s) [seg_pos={0 0}, upload=371666]: Begin upload segment...
Aug 23 14:45:24.512685454 (357h19m18.943765585s) [seg_pos={0 0}, upload=371666]: Done begin upload segment.
Aug 23 14:45:24.512705382 (357h19m18.943785513s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.512739748 (357h19m18.943819879s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513073596 (357h19m18.944153727s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513189728 (357h19m18.944269859s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513374562 (357h19m18.944454693s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513700065 (357h19m18.944780206s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514065845 (357h19m18.945145976s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514450741 (357h19m18.945530872s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514690520 (357h19m18.945770651s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514915771 (357h19m18.945995902s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515141302 (357h19m18.946221433s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515335314 (357h19m18.946415446s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515543913 (357h19m18.946624044s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515757332 (357h19m18.946837463s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515967714 (357h19m18.947047835s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516176834 (357h19m18.947256965s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516377859 (357h19m18.94745798s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516614882 (357h19m18.947695013s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:26.051354502 (357h19m20.482434633s) [upload=371666]: Next returned nil segment
Aug 23 14:45:26.051372857 (357h19m20.482452988s) [upload=371666]: Waiting for error group managing segments...
Aug 23 14:45:26.499241712 (357h19m20.930321843s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.532419114 (357h19m20.963499245s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.628848182 (357h19m21.059928313s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.666022381 (357h19m21.097102522s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.668507786 (357h19m21.099587917s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.717353037 (357h19m21.148433168s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.748823854 (357h19m21.179903995s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.763937462 (357h19m21.195017593s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.769131070 (357h19m21.200211201s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.785056963 (357h19m21.216137094s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.786918465 (357h19m21.217998596s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.853343923 (357h19m21.284424054s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.879991480 (357h19m21.311071611s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889610165 (357h19m21.320690296s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889676141 (357h19m21.320756272s) [seg_pos={0 0}, upload=371666]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:45:26.890072289 (357h19m21.32115242s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890132134 (357h19m21.321212265s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890158935 (357h19m21.321239066s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE): protocol: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection
Aug 23 14:45:26.890228298 (357h19m21.321308419s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct): protocol: rpc: dial tcp 104.36.82.23:4472: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 23 14:45:26.890253176 (357h19m21.321333317s) [seg_pos={0 0}, upload=371666]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:45:26.890333268 (357h19m21.321413399s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:45:26.890338609 (357h19m21.32141874s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:45:27.223463992 (357h19m21.654544124s) [upload=371666]: Done uploading segments...
Aug 23 14:45:27.223483129 (357h19m21.65456326s) [upload=371666]: Upload finished. err: <nil>
Duration: 6.614700278s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.707412129s
FAIL (2025-08-23 14:48:18 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Aug 23 14:06:34.087047198 (356h40m28.518127319s) [upload=371002]: Starting upload
Aug 23 14:06:34.087103405 (356h40m28.518183536s) [upload=371002]: Done starting upload
Aug 23 14:06:34.087217894 (356h40m28.518298075s) [upload=371002]: Uploading segments...
Aug 23 14:06:34.087280084 (356h40m28.518360225s) [upload=371002]: Got next segment. Inline: false
Aug 23 14:06:34.087333556 (356h40m28.518413697s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:06:34.087345218 (356h40m28.518425379s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:06:34.329714999 (356h40m28.76079513s) [seg_pos={0 0}, upload=371002]: Begin upload segment...
Aug 23 14:06:34.329766266 (356h40m28.760846387s) [seg_pos={0 0}, upload=371002]: Done begin upload segment.
Aug 23 14:06:34.329787617 (356h40m28.760867758s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.329825380 (356h40m28.760905521s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330128550 (356h40m28.761208681s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330176731 (356h40m28.761256862s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330343832 (356h40m28.761423953s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330470363 (356h40m28.761550494s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330760569 (356h40m28.76184071s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331052808 (356h40m28.762132939s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331345137 (356h40m28.762425268s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331803835 (356h40m28.762883966s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332093870 (356h40m28.763174001s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332431015 (356h40m28.763511146s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332721310 (356h40m28.763801441s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333000254 (356h40m28.764080385s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333290029 (356h40m28.76437016s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333579212 (356h40m28.764659343s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:36.088000547 (356h40m30.519080668s) [upload=371002]: Next returned nil segment
Aug 23 14:06:36.088019162 (356h40m30.519099293s) [upload=371002]: Waiting for error group managing segments...
Aug 23 14:06:36.365114618 (356h40m30.79619475s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.384930313 (356h40m30.816010444s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.441019685 (356h40m30.872099816s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.503003134 (356h40m30.934083265s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.773706448 (356h40m31.204786629s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.794301936 (356h40m31.225382068s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.797422086 (356h40m31.228502217s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.891493522 (356h40m31.322573653s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.893130365 (356h40m31.324210496s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.926291877 (356h40m31.357372008s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.980332537 (356h40m31.411412668s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.986328430 (356h40m31.417408551s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126158031 (356h40m31.557238152s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126185253 (356h40m31.557265384s) [seg_pos={0 0}, upload=371002]: Segment reached optimal threshold of 13 pieces.
Aug 23 14:06:37.126290142 (356h40m31.557370274s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126445190 (356h40m31.557525321s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection
Aug 23 14:06:37.126492731 (356h40m31.557572862s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126560811 (356h40m31.557640952s) [seg_pos={0 0}, upload=371002]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 23 14:06:37.126686101 (356h40m31.557766232s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:06:37.126693204 (356h40m31.557773365s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:06:37.513830133 (356h40m31.944910264s) [upload=371002]: Done uploading segments...
Aug 23 14:06:37.513848849 (356h40m31.944928981s) [upload=371002]: Upload finished. err: <nil>
Duration: 6.707436818s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...........................X.......................X........] duration:
1.933601325s
FAIL (2025-08-23 14:47:53 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.93362964s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.........X..................X....X.X....................X...] duration:
6.446911116s
FAIL (2025-08-23 14:47:45 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:45:24.054384699 (357h19m18.48546484s) [upload=371666]: Starting upload
Aug 23 14:45:24.054471174 (357h19m18.485551305s) [upload=371666]: Done starting upload
Aug 23 14:45:24.054563761 (357h19m18.485643892s) [upload=371666]: Uploading segments...
Aug 23 14:45:24.054700883 (357h19m18.485781014s) [upload=371666]: Got next segment. Inline: false
Aug 23 14:45:24.054758313 (357h19m18.485838484s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:45:24.054767090 (357h19m18.485847261s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:45:24.512644204 (357h19m18.943724336s) [seg_pos={0 0}, upload=371666]: Begin upload segment...
Aug 23 14:45:24.512685454 (357h19m18.943765585s) [seg_pos={0 0}, upload=371666]: Done begin upload segment.
Aug 23 14:45:24.512705382 (357h19m18.943785513s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.512739748 (357h19m18.943819879s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513073596 (357h19m18.944153727s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513189728 (357h19m18.944269859s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513374562 (357h19m18.944454693s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513700065 (357h19m18.944780206s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514065845 (357h19m18.945145976s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514450741 (357h19m18.945530872s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514690520 (357h19m18.945770651s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514915771 (357h19m18.945995902s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515141302 (357h19m18.946221433s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515335314 (357h19m18.946415446s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515543913 (357h19m18.946624044s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515757332 (357h19m18.946837463s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515967714 (357h19m18.947047835s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516176834 (357h19m18.947256965s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516377859 (357h19m18.94745798s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516614882 (357h19m18.947695013s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:26.051354502 (357h19m20.482434633s) [upload=371666]: Next returned nil segment
Aug 23 14:45:26.051372857 (357h19m20.482452988s) [upload=371666]: Waiting for error group managing segments...
Aug 23 14:45:26.499241712 (357h19m20.930321843s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.532419114 (357h19m20.963499245s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.628848182 (357h19m21.059928313s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.666022381 (357h19m21.097102522s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.668507786 (357h19m21.099587917s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.717353037 (357h19m21.148433168s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.748823854 (357h19m21.179903995s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.763937462 (357h19m21.195017593s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.769131070 (357h19m21.200211201s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.785056963 (357h19m21.216137094s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.786918465 (357h19m21.217998596s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.853343923 (357h19m21.284424054s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.879991480 (357h19m21.311071611s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889610165 (357h19m21.320690296s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889676141 (357h19m21.320756272s) [seg_pos={0 0}, upload=371666]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:45:26.890072289 (357h19m21.32115242s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890132134 (357h19m21.321212265s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890158935 (357h19m21.321239066s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE): protocol: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection
Aug 23 14:45:26.890228298 (357h19m21.321308419s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct): protocol: rpc: dial tcp 104.36.82.23:4472: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 23 14:45:26.890253176 (357h19m21.321333317s) [seg_pos={0 0}, upload=371666]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:45:26.890333268 (357h19m21.321413399s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:45:26.890338609 (357h19m21.32141874s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:45:27.223463992 (357h19m21.654544124s) [upload=371666]: Done uploading segments...
Aug 23 14:45:27.223483129 (357h19m21.65456326s) [upload=371666]: Upload finished. err: <nil>
Duration: 6.446934029s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
6.643506874s
FAIL (2025-08-23 14:47:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Aug 23 14:06:34.087047198 (356h40m28.518127319s) [upload=371002]: Starting upload
Aug 23 14:06:34.087103405 (356h40m28.518183536s) [upload=371002]: Done starting upload
Aug 23 14:06:34.087217894 (356h40m28.518298075s) [upload=371002]: Uploading segments...
Aug 23 14:06:34.087280084 (356h40m28.518360225s) [upload=371002]: Got next segment. Inline: false
Aug 23 14:06:34.087333556 (356h40m28.518413697s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:06:34.087345218 (356h40m28.518425379s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:06:34.329714999 (356h40m28.76079513s) [seg_pos={0 0}, upload=371002]: Begin upload segment...
Aug 23 14:06:34.329766266 (356h40m28.760846387s) [seg_pos={0 0}, upload=371002]: Done begin upload segment.
Aug 23 14:06:34.329787617 (356h40m28.760867758s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.329825380 (356h40m28.760905521s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330128550 (356h40m28.761208681s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330176731 (356h40m28.761256862s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330343832 (356h40m28.761423953s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330470363 (356h40m28.761550494s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330760569 (356h40m28.76184071s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331052808 (356h40m28.762132939s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331345137 (356h40m28.762425268s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331803835 (356h40m28.762883966s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332093870 (356h40m28.763174001s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332431015 (356h40m28.763511146s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332721310 (356h40m28.763801441s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333000254 (356h40m28.764080385s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333290029 (356h40m28.76437016s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333579212 (356h40m28.764659343s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:36.088000547 (356h40m30.519080668s) [upload=371002]: Next returned nil segment
Aug 23 14:06:36.088019162 (356h40m30.519099293s) [upload=371002]: Waiting for error group managing segments...
Aug 23 14:06:36.365114618 (356h40m30.79619475s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.384930313 (356h40m30.816010444s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.441019685 (356h40m30.872099816s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.503003134 (356h40m30.934083265s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.773706448 (356h40m31.204786629s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.794301936 (356h40m31.225382068s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.797422086 (356h40m31.228502217s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.891493522 (356h40m31.322573653s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.893130365 (356h40m31.324210496s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.926291877 (356h40m31.357372008s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.980332537 (356h40m31.411412668s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.986328430 (356h40m31.417408551s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126158031 (356h40m31.557238152s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126185253 (356h40m31.557265384s) [seg_pos={0 0}, upload=371002]: Segment reached optimal threshold of 13 pieces.
Aug 23 14:06:37.126290142 (356h40m31.557370274s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126445190 (356h40m31.557525321s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection
Aug 23 14:06:37.126492731 (356h40m31.557572862s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126560811 (356h40m31.557640952s) [seg_pos={0 0}, upload=371002]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 23 14:06:37.126686101 (356h40m31.557766232s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:06:37.126693204 (356h40m31.557773365s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:06:37.513830133 (356h40m31.944910264s) [upload=371002]: Done uploading segments...
Aug 23 14:06:37.513848849 (356h40m31.944928981s) [upload=371002]: Upload finished. err: <nil>
Duration: 6.643539025s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [X.........X..................X....X.X....................X..] duration:
6.17129999s
FAIL (2025-08-23 14:47:06 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:45:24.054384699 (357h19m18.48546484s) [upload=371666]: Starting upload
Aug 23 14:45:24.054471174 (357h19m18.485551305s) [upload=371666]: Done starting upload
Aug 23 14:45:24.054563761 (357h19m18.485643892s) [upload=371666]: Uploading segments...
Aug 23 14:45:24.054700883 (357h19m18.485781014s) [upload=371666]: Got next segment. Inline: false
Aug 23 14:45:24.054758313 (357h19m18.485838484s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:45:24.054767090 (357h19m18.485847261s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:45:24.512644204 (357h19m18.943724336s) [seg_pos={0 0}, upload=371666]: Begin upload segment...
Aug 23 14:45:24.512685454 (357h19m18.943765585s) [seg_pos={0 0}, upload=371666]: Done begin upload segment.
Aug 23 14:45:24.512705382 (357h19m18.943785513s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.512739748 (357h19m18.943819879s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513073596 (357h19m18.944153727s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513189728 (357h19m18.944269859s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513374562 (357h19m18.944454693s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513700065 (357h19m18.944780206s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514065845 (357h19m18.945145976s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514450741 (357h19m18.945530872s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514690520 (357h19m18.945770651s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514915771 (357h19m18.945995902s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515141302 (357h19m18.946221433s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515335314 (357h19m18.946415446s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515543913 (357h19m18.946624044s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515757332 (357h19m18.946837463s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515967714 (357h19m18.947047835s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516176834 (357h19m18.947256965s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516377859 (357h19m18.94745798s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516614882 (357h19m18.947695013s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:26.051354502 (357h19m20.482434633s) [upload=371666]: Next returned nil segment
Aug 23 14:45:26.051372857 (357h19m20.482452988s) [upload=371666]: Waiting for error group managing segments...
Aug 23 14:45:26.499241712 (357h19m20.930321843s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.532419114 (357h19m20.963499245s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.628848182 (357h19m21.059928313s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.666022381 (357h19m21.097102522s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.668507786 (357h19m21.099587917s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.717353037 (357h19m21.148433168s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.748823854 (357h19m21.179903995s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.763937462 (357h19m21.195017593s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.769131070 (357h19m21.200211201s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.785056963 (357h19m21.216137094s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.786918465 (357h19m21.217998596s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.853343923 (357h19m21.284424054s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.879991480 (357h19m21.311071611s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889610165 (357h19m21.320690296s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889676141 (357h19m21.320756272s) [seg_pos={0 0}, upload=371666]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:45:26.890072289 (357h19m21.32115242s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890132134 (357h19m21.321212265s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890158935 (357h19m21.321239066s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE): protocol: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection
Aug 23 14:45:26.890228298 (357h19m21.321308419s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct): protocol: rpc: dial tcp 104.36.82.23:4472: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 23 14:45:26.890253176 (357h19m21.321333317s) [seg_pos={0 0}, upload=371666]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:45:26.890333268 (357h19m21.321413399s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:45:26.890338609 (357h19m21.32141874s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:45:27.223463992 (357h19m21.654544124s) [upload=371666]: Done uploading segments...
Aug 23 14:45:27.223483129 (357h19m21.65456326s) [upload=371666]: Upload finished. err: <nil>
Duration: 6.171328364s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
6.669272425s
FAIL (2025-08-23 14:46:56 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Aug 23 14:06:34.087047198 (356h40m28.518127319s) [upload=371002]: Starting upload
Aug 23 14:06:34.087103405 (356h40m28.518183536s) [upload=371002]: Done starting upload
Aug 23 14:06:34.087217894 (356h40m28.518298075s) [upload=371002]: Uploading segments...
Aug 23 14:06:34.087280084 (356h40m28.518360225s) [upload=371002]: Got next segment. Inline: false
Aug 23 14:06:34.087333556 (356h40m28.518413697s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:06:34.087345218 (356h40m28.518425379s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:06:34.329714999 (356h40m28.76079513s) [seg_pos={0 0}, upload=371002]: Begin upload segment...
Aug 23 14:06:34.329766266 (356h40m28.760846387s) [seg_pos={0 0}, upload=371002]: Done begin upload segment.
Aug 23 14:06:34.329787617 (356h40m28.760867758s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.329825380 (356h40m28.760905521s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330128550 (356h40m28.761208681s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330176731 (356h40m28.761256862s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330343832 (356h40m28.761423953s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330470363 (356h40m28.761550494s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330760569 (356h40m28.76184071s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331052808 (356h40m28.762132939s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331345137 (356h40m28.762425268s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331803835 (356h40m28.762883966s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332093870 (356h40m28.763174001s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332431015 (356h40m28.763511146s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332721310 (356h40m28.763801441s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333000254 (356h40m28.764080385s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333290029 (356h40m28.76437016s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333579212 (356h40m28.764659343s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:36.088000547 (356h40m30.519080668s) [upload=371002]: Next returned nil segment
Aug 23 14:06:36.088019162 (356h40m30.519099293s) [upload=371002]: Waiting for error group managing segments...
Aug 23 14:06:36.365114618 (356h40m30.79619475s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.384930313 (356h40m30.816010444s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.441019685 (356h40m30.872099816s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.503003134 (356h40m30.934083265s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.773706448 (356h40m31.204786629s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.794301936 (356h40m31.225382068s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.797422086 (356h40m31.228502217s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.891493522 (356h40m31.322573653s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.893130365 (356h40m31.324210496s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.926291877 (356h40m31.357372008s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.980332537 (356h40m31.411412668s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.986328430 (356h40m31.417408551s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126158031 (356h40m31.557238152s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126185253 (356h40m31.557265384s) [seg_pos={0 0}, upload=371002]: Segment reached optimal threshold of 13 pieces.
Aug 23 14:06:37.126290142 (356h40m31.557370274s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126445190 (356h40m31.557525321s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection
Aug 23 14:06:37.126492731 (356h40m31.557572862s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126560811 (356h40m31.557640952s) [seg_pos={0 0}, upload=371002]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 23 14:06:37.126686101 (356h40m31.557766232s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:06:37.126693204 (356h40m31.557773365s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:06:37.513830133 (356h40m31.944910264s) [upload=371002]: Done uploading segments...
Aug 23 14:06:37.513848849 (356h40m31.944928981s) [upload=371002]: Upload finished. err: <nil>
Duration: 6.669297252s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [............................X.......................X.......] duration:
1.977836833s
FAIL (2025-08-23 14:46:55 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.977864307s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
6.889651374s
FAIL (2025-08-23 14:46:33 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Aug 23 14:06:34.087047198 (356h40m28.518127319s) [upload=371002]: Starting upload
Aug 23 14:06:34.087103405 (356h40m28.518183536s) [upload=371002]: Done starting upload
Aug 23 14:06:34.087217894 (356h40m28.518298075s) [upload=371002]: Uploading segments...
Aug 23 14:06:34.087280084 (356h40m28.518360225s) [upload=371002]: Got next segment. Inline: false
Aug 23 14:06:34.087333556 (356h40m28.518413697s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:06:34.087345218 (356h40m28.518425379s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:06:34.329714999 (356h40m28.76079513s) [seg_pos={0 0}, upload=371002]: Begin upload segment...
Aug 23 14:06:34.329766266 (356h40m28.760846387s) [seg_pos={0 0}, upload=371002]: Done begin upload segment.
Aug 23 14:06:34.329787617 (356h40m28.760867758s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.329825380 (356h40m28.760905521s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330128550 (356h40m28.761208681s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330176731 (356h40m28.761256862s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330343832 (356h40m28.761423953s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330470363 (356h40m28.761550494s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330760569 (356h40m28.76184071s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331052808 (356h40m28.762132939s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331345137 (356h40m28.762425268s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331803835 (356h40m28.762883966s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332093870 (356h40m28.763174001s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332431015 (356h40m28.763511146s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332721310 (356h40m28.763801441s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333000254 (356h40m28.764080385s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333290029 (356h40m28.76437016s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333579212 (356h40m28.764659343s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:36.088000547 (356h40m30.519080668s) [upload=371002]: Next returned nil segment
Aug 23 14:06:36.088019162 (356h40m30.519099293s) [upload=371002]: Waiting for error group managing segments...
Aug 23 14:06:36.365114618 (356h40m30.79619475s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.384930313 (356h40m30.816010444s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.441019685 (356h40m30.872099816s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.503003134 (356h40m30.934083265s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.773706448 (356h40m31.204786629s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.794301936 (356h40m31.225382068s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.797422086 (356h40m31.228502217s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.891493522 (356h40m31.322573653s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.893130365 (356h40m31.324210496s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.926291877 (356h40m31.357372008s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.980332537 (356h40m31.411412668s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.986328430 (356h40m31.417408551s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126158031 (356h40m31.557238152s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126185253 (356h40m31.557265384s) [seg_pos={0 0}, upload=371002]: Segment reached optimal threshold of 13 pieces.
Aug 23 14:06:37.126290142 (356h40m31.557370274s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126445190 (356h40m31.557525321s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection
Aug 23 14:06:37.126492731 (356h40m31.557572862s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126560811 (356h40m31.557640952s) [seg_pos={0 0}, upload=371002]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 23 14:06:37.126686101 (356h40m31.557766232s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:06:37.126693204 (356h40m31.557773365s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:06:37.513830133 (356h40m31.944910264s) [upload=371002]: Done uploading segments...
Aug 23 14:06:37.513848849 (356h40m31.944928981s) [upload=371002]: Upload finished. err: <nil>
Duration: 6.889686451s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............................X.......................X......] duration:
1.965894363s
FAIL (2025-08-23 14:46:32 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.9659191s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.X.........X..................X....X.X....................X.] duration:
6.199092689s
FAIL (2025-08-23 14:46:13 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:45:24.054384699 (357h19m18.48546484s) [upload=371666]: Starting upload
Aug 23 14:45:24.054471174 (357h19m18.485551305s) [upload=371666]: Done starting upload
Aug 23 14:45:24.054563761 (357h19m18.485643892s) [upload=371666]: Uploading segments...
Aug 23 14:45:24.054700883 (357h19m18.485781014s) [upload=371666]: Got next segment. Inline: false
Aug 23 14:45:24.054758313 (357h19m18.485838484s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:45:24.054767090 (357h19m18.485847261s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:45:24.512644204 (357h19m18.943724336s) [seg_pos={0 0}, upload=371666]: Begin upload segment...
Aug 23 14:45:24.512685454 (357h19m18.943765585s) [seg_pos={0 0}, upload=371666]: Done begin upload segment.
Aug 23 14:45:24.512705382 (357h19m18.943785513s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.512739748 (357h19m18.943819879s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513073596 (357h19m18.944153727s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513189728 (357h19m18.944269859s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513374562 (357h19m18.944454693s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513700065 (357h19m18.944780206s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514065845 (357h19m18.945145976s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514450741 (357h19m18.945530872s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514690520 (357h19m18.945770651s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514915771 (357h19m18.945995902s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515141302 (357h19m18.946221433s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515335314 (357h19m18.946415446s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515543913 (357h19m18.946624044s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515757332 (357h19m18.946837463s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515967714 (357h19m18.947047835s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516176834 (357h19m18.947256965s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516377859 (357h19m18.94745798s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516614882 (357h19m18.947695013s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:26.051354502 (357h19m20.482434633s) [upload=371666]: Next returned nil segment
Aug 23 14:45:26.051372857 (357h19m20.482452988s) [upload=371666]: Waiting for error group managing segments...
Aug 23 14:45:26.499241712 (357h19m20.930321843s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.532419114 (357h19m20.963499245s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.628848182 (357h19m21.059928313s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.666022381 (357h19m21.097102522s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.668507786 (357h19m21.099587917s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.717353037 (357h19m21.148433168s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.748823854 (357h19m21.179903995s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.763937462 (357h19m21.195017593s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.769131070 (357h19m21.200211201s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.785056963 (357h19m21.216137094s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.786918465 (357h19m21.217998596s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.853343923 (357h19m21.284424054s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.879991480 (357h19m21.311071611s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889610165 (357h19m21.320690296s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889676141 (357h19m21.320756272s) [seg_pos={0 0}, upload=371666]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:45:26.890072289 (357h19m21.32115242s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890132134 (357h19m21.321212265s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890158935 (357h19m21.321239066s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE): protocol: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection
Aug 23 14:45:26.890228298 (357h19m21.321308419s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct): protocol: rpc: dial tcp 104.36.82.23:4472: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 23 14:45:26.890253176 (357h19m21.321333317s) [seg_pos={0 0}, upload=371666]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:45:26.890333268 (357h19m21.321413399s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:45:26.890338609 (357h19m21.32141874s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:45:27.223463992 (357h19m21.654544124s) [upload=371666]: Done uploading segments...
Aug 23 14:45:27.223483129 (357h19m21.65456326s) [upload=371666]: Upload finished. err: <nil>
Duration: 6.199113308s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............................X.......................X.....] duration:
1.990310982s
FAIL (2025-08-23 14:45:50 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.990340258s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
6.652985946s
FAIL (2025-08-23 14:45:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Aug 23 14:06:34.087047198 (356h40m28.518127319s) [upload=371002]: Starting upload
Aug 23 14:06:34.087103405 (356h40m28.518183536s) [upload=371002]: Done starting upload
Aug 23 14:06:34.087217894 (356h40m28.518298075s) [upload=371002]: Uploading segments...
Aug 23 14:06:34.087280084 (356h40m28.518360225s) [upload=371002]: Got next segment. Inline: false
Aug 23 14:06:34.087333556 (356h40m28.518413697s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:06:34.087345218 (356h40m28.518425379s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:06:34.329714999 (356h40m28.76079513s) [seg_pos={0 0}, upload=371002]: Begin upload segment...
Aug 23 14:06:34.329766266 (356h40m28.760846387s) [seg_pos={0 0}, upload=371002]: Done begin upload segment.
Aug 23 14:06:34.329787617 (356h40m28.760867758s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.329825380 (356h40m28.760905521s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330128550 (356h40m28.761208681s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330176731 (356h40m28.761256862s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330343832 (356h40m28.761423953s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330470363 (356h40m28.761550494s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330760569 (356h40m28.76184071s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331052808 (356h40m28.762132939s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331345137 (356h40m28.762425268s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331803835 (356h40m28.762883966s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332093870 (356h40m28.763174001s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332431015 (356h40m28.763511146s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332721310 (356h40m28.763801441s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333000254 (356h40m28.764080385s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333290029 (356h40m28.76437016s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333579212 (356h40m28.764659343s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:36.088000547 (356h40m30.519080668s) [upload=371002]: Next returned nil segment
Aug 23 14:06:36.088019162 (356h40m30.519099293s) [upload=371002]: Waiting for error group managing segments...
Aug 23 14:06:36.365114618 (356h40m30.79619475s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.384930313 (356h40m30.816010444s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.441019685 (356h40m30.872099816s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.503003134 (356h40m30.934083265s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.773706448 (356h40m31.204786629s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.794301936 (356h40m31.225382068s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.797422086 (356h40m31.228502217s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.891493522 (356h40m31.322573653s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.893130365 (356h40m31.324210496s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.926291877 (356h40m31.357372008s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.980332537 (356h40m31.411412668s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.986328430 (356h40m31.417408551s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126158031 (356h40m31.557238152s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126185253 (356h40m31.557265384s) [seg_pos={0 0}, upload=371002]: Segment reached optimal threshold of 13 pieces.
Aug 23 14:06:37.126290142 (356h40m31.557370274s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126445190 (356h40m31.557525321s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection
Aug 23 14:06:37.126492731 (356h40m31.557572862s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126560811 (356h40m31.557640952s) [seg_pos={0 0}, upload=371002]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 23 14:06:37.126686101 (356h40m31.557766232s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:06:37.126693204 (356h40m31.557773365s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:06:37.513830133 (356h40m31.944910264s) [upload=371002]: Done uploading segments...
Aug 23 14:06:37.513848849 (356h40m31.944928981s) [upload=371002]: Upload finished. err: <nil>
Duration: 6.653015041s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [..X.........X..................X....X.X....................X] duration:
4.495535534s
FAIL (2025-08-23 14:45:27 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 23 14:45:24.054384699 (357h19m18.48546484s) [upload=371666]: Starting upload
Aug 23 14:45:24.054471174 (357h19m18.485551305s) [upload=371666]: Done starting upload
Aug 23 14:45:24.054563761 (357h19m18.485643892s) [upload=371666]: Uploading segments...
Aug 23 14:45:24.054700883 (357h19m18.485781014s) [upload=371666]: Got next segment. Inline: false
Aug 23 14:45:24.054758313 (357h19m18.485838484s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:45:24.054767090 (357h19m18.485847261s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:45:24.512644204 (357h19m18.943724336s) [seg_pos={0 0}, upload=371666]: Begin upload segment...
Aug 23 14:45:24.512685454 (357h19m18.943765585s) [seg_pos={0 0}, upload=371666]: Done begin upload segment.
Aug 23 14:45:24.512705382 (357h19m18.943785513s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.512739748 (357h19m18.943819879s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513073596 (357h19m18.944153727s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513189728 (357h19m18.944269859s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513374562 (357h19m18.944454693s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.513700065 (357h19m18.944780206s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514065845 (357h19m18.945145976s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514450741 (357h19m18.945530872s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514690520 (357h19m18.945770651s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.514915771 (357h19m18.945995902s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515141302 (357h19m18.946221433s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515335314 (357h19m18.946415446s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515543913 (357h19m18.946624044s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515757332 (357h19m18.946837463s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.515967714 (357h19m18.947047835s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516176834 (357h19m18.947256965s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516377859 (357h19m18.94745798s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:24.516614882 (357h19m18.947695013s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Uploading piece...
Aug 23 14:45:26.051354502 (357h19m20.482434633s) [upload=371666]: Next returned nil segment
Aug 23 14:45:26.051372857 (357h19m20.482452988s) [upload=371666]: Waiting for error group managing segments...
Aug 23 14:45:26.499241712 (357h19m20.930321843s) [noise=true , address=79.127.219.47:4072 , piece_id=5KHBQUYVRT5I46BRO35UNTKKCMOS2KHHAMIJC6ZMBWFGZT3WTLLA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.532419114 (357h19m20.963499245s) [noise=true , address=79.127.219.48:4099 , piece_id=OOVC2ZOY2XGDGJBWNKQX7HEZNOKSQT43M3PGK6JFZWT5LJ6GA7WQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.628848182 (357h19m21.059928313s) [noise=true , address=79.127.219.51:4209 , piece_id=EDFOXGLJW57UAZUFKXSYJPE3HWCIQQBTGKSHXEYFRADLOSEOLQAA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.666022381 (357h19m21.097102522s) [noise=true , address=104.36.82.3:4787 , piece_id=GSURUU3GRWWQXC4IK4FFY5RQHJH3BY2S5UKOAOYJQERTW7373YVQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.668507786 (357h19m21.099587917s) [noise=true , address=104.36.82.44:4879 , piece_id=ORZWZHFHAKMMJ5GNIGZUAJIXTHK4CUEVW4DLBACKRLEVZVLYUONQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.717353037 (357h19m21.148433168s) [noise=true , address=104.36.82.50:4043 , piece_id=WTD5CDNNNGUVWXSYXCFJNUDRMIUJ6VFLGFT5SED66G5P5ALH4XWA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.748823854 (357h19m21.179903995s) [noise=true , address=77.90.31.19:4865 , piece_id=S5XV7VEMODQV5DM2UNL3SFRUKGTLTAGBKGM54WWU7E5RTMCWOQCQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.763937462 (357h19m21.195017593s) [noise=true , address=104.36.82.60:4339 , piece_id=GM3LUTJZLLSNYBV4T23A4ZNMPU3JE6NWIQ6YMUZQGBAAANKL7JIA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.769131070 (357h19m21.200211201s) [noise=true , address=45.91.116.14:4207 , piece_id=Y7SEVLURPFA5N3NGYMEYDSEYMR5QPECKNZHORNSLMSQLIYO5CIXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.785056963 (357h19m21.216137094s) [noise=true , address=77.90.31.12:4621 , piece_id=7OTASWJSXVCCSQQ6UD4BLOT6ZXTXAUNHEXKJGMAHJUWHSDPLQ3SQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.786918465 (357h19m21.217998596s) [noise=true , address=77.90.31.11:4559 , piece_id=7NRPD6VQYS5GN52LS5DIKN3B7JKZKXPZJEX3PWZKRBCML7SYVMXQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.853343923 (357h19m21.284424054s) [noise=true , address=77.90.31.10:4525 , piece_id=CDZZZA7WUNOTDITPVFPVA24DJ37HV3F765II7HZMIZVLVMWOU5TA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.879991480 (357h19m21.311071611s) [noise=true , address=45.91.116.10:4108 , piece_id=6R4YERAR2OLKFGTD5HSXDWHYDOZPHMOES45OL2XE2OL45ZDHDIIQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889610165 (357h19m21.320690296s) [noise=true , address=45.91.116.12:4253 , piece_id=LJUF4RIHLC2VHXHCCDSZXBEEKPWAN3CX66EH7ID6LNFQVCIOBPSA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: <nil>
Aug 23 14:45:26.889676141 (357h19m21.320756272s) [seg_pos={0 0}, upload=371666]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:45:26.890072289 (357h19m21.32115242s) [noise=true , address=77.90.31.16:4740 , piece_id=J7JEISVLPSNY3HIVGMCI3EYMNOKGYGSNPW36RB563B3ZX6MIIR4Q, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890132134 (357h19m21.321212265s) [noise=true , address=77.90.31.18:4837 , piece_id=5UFSZUSAAASL7ODOVUNXRWH27NDUVHHPKBVVN7JVUEWQZ3R3TDRQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j): protocol: expected piece hash; context canceled; EOF
Aug 23 14:45:26.890158935 (357h19m21.321239066s) [noise=true , address=104.36.82.5:4860 , piece_id=DXWYPJGNGYWJY7DAMPGFTOOKPKIG6SUT2HLIJNDMPTEPRDLQCVDQ, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE): protocol: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52478->104.36.82.5:4860: use of closed network connection
Aug 23 14:45:26.890228298 (357h19m21.321308419s) [noise=true , address=104.36.82.23:4472 , piece_id=O7OUNBFZ6CYBNOCRVUVTT3GWZRMTIVJGLG5FXBMKKCBKFJIK5IEA, seg_pos={0 0}, upload=371666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct): protocol: rpc: dial tcp 104.36.82.23:4472: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 23 14:45:26.890253176 (357h19m21.321333317s) [seg_pos={0 0}, upload=371666]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:45:26.890333268 (357h19m21.321413399s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:45:26.890338609 (357h19m21.32141874s) [upload=371666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:45:27.223463992 (357h19m21.654544124s) [upload=371666]: Done uploading segments...
Aug 23 14:45:27.223483129 (357h19m21.65456326s) [upload=371666]: Upload finished. err: <nil>
Duration: 4.495569428s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...X.........X..................X....X.X....................] duration:
6.397038871s
FAIL (2025-08-23 14:44:56 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: invalid range 0:4194304 (size:4194304)
======LOGS======
Aug 23 14:27:59.916879691 (357h1m54.347959822s) [upload=371367]: Starting upload
Aug 23 14:27:59.916927211 (357h1m54.348007342s) [upload=371367]: Done starting upload
Aug 23 14:27:59.916964042 (357h1m54.348044183s) [upload=371367]: Uploading segments...
Aug 23 14:27:59.917143525 (357h1m54.348223656s) [upload=371367]: Got next segment. Inline: false
Aug 23 14:27:59.917178773 (357h1m54.348258894s) [upload=371367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:27:59.917183542 (357h1m54.348263663s) [upload=371367]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:28:00.381688206 (357h1m54.812768337s) [seg_pos={0 0}, upload=371367]: Begin upload segment...
Aug 23 14:28:00.381881216 (357h1m54.812961357s) [seg_pos={0 0}, upload=371367]: Done begin upload segment.
Aug 23 14:28:00.381921983 (357h1m54.813002114s) [noise=true , address=79.127.205.234:4596 , piece_id=VWEQWYDCV36AHJAQBB7IIH2GNU6UGOQCKOYMKQQH5M3BF5ZAMWXQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.381955859 (357h1m54.813036s) [noise=true , address=104.36.82.60:4344 , piece_id=XPVIM4N4OQJTGECEY7G7CDBKK6GXUAR5W7B32MQXHY57RDGT5O5Q, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.382447860 (357h1m54.813527991s) [noise=true , address=104.36.82.44:4891 , piece_id=FLVR7XEMS2ZEIDSX4XNYEXMCLXXV63KVFOLVJRQVKZLN2LWUWQXQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.382500691 (357h1m54.813580822s) [noise=true , address=79.127.201.215:4343 , piece_id=JNDYG6ZMDU5U6LEF5UBJCGAWVATWRYNELDQAZG35UQ3XTXF5ZSQA, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.382775156 (357h1m54.813855288s) [noise=true , address=77.90.31.11:4567 , piece_id=GDL3HJZW2BZ3Y3LUI2SNWJGBBJEBVRO4XQBJWDUE75JZUEDGP32Q, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.382817186 (357h1m54.813897317s) [noise=true , address=77.90.31.13:4649 , piece_id=RFC7SITGS37TZH7LQWMQ7IYBOLARTCSKF6HSLDX5XGJ7OXL7PPEQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.383157026 (357h1m54.814237157s) [noise=true , address=77.90.31.16:4767 , piece_id=4FI74HXN3LRRJ467GI3RVQSJOBPXIJFYZOT3KSX5KO2ZIMK63O6A, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.383182946 (357h1m54.814263077s) [noise=true , address=77.90.31.18:4815 , piece_id=JMB6SIQUN4DN3R23NMJMPIRKLCYVJFOK7LUO5KYYNF3FVO7QAWLA, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.383562633 (357h1m54.814642774s) [noise=true , address=77.90.31.10:4543 , piece_id=OPTZPP5SMR7SOL74XHVJJFPZPLTAJCAUEJWX3SZYZJZ2UAIMYPPQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.383910348 (357h1m54.814990479s) [noise=true , address=79.127.219.47:4063 , piece_id=AJHVIZ4TASKYYK525R6KCJU7JEFAVPL6RQTJX53DSQEDZBH5GBFQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.384252182 (357h1m54.815332313s) [noise=true , address=79.127.219.48:4091 , piece_id=ZZN4NXJIRQRSKZZRXIZN4A4ZY7XAZU4WOZQ2773BFZU6R5GWQ5QA, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.384635074 (357h1m54.815715205s) [noise=true , address=104.36.82.19:4417 , piece_id=ZQIMDN66BBNEVH7FHX6PX2R4CZPULEWJDLOOZIQXAH7P2ABJSIDQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.384965086 (357h1m54.816045217s) [noise=true , address=104.36.82.52:4119 , piece_id=RHS5Q42HARU5X4NAEPGDEUGYSS64LUARZIVAAXIU3BD2AJSUZ3TQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.385316959 (357h1m54.81639709s) [noise=true , address=77.90.31.19:4857 , piece_id=KOGKADKRYREAULHF54Y67DYM4V3L7V4AL6EHDVBDWMHYJNKYMLSA, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.385618246 (357h1m54.816698387s) [noise=true , address=79.127.219.51:4212 , piece_id=I4A5G4NB5LZGZ4AKS4ZJXAX6JWFX7I4XDTZGZQ7BTJVGTWHQWQTQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.385957936 (357h1m54.817038067s) [noise=true , address=77.90.31.12:4624 , piece_id=H6PI6BL7LMSXNBLA3RR4UVNRU5KHOJXC3SBBJ3NBBZDSJQQ5NF2Q, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.386250415 (357h1m54.817330566s) [noise=true , address=104.36.82.20:4461 , piece_id=NGNQCJQKJD32YX5GWV2VHJCW6XMMCGMUW3OYBC5OQSHVTYFQE6WA, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.386596428 (357h1m54.817676569s) [noise=true , address=104.36.82.5:4861 , piece_id=TLUVB7AWEOWHPVNRQGIQ3V6GDSL4AIIEVVEXCMBLJIHIW3HYUTOQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:01.551634527 (357h1m55.982714658s) [upload=371367]: Next returned nil segment
Aug 23 14:28:01.551654415 (357h1m55.982734556s) [upload=371367]: Waiting for error group managing segments...
Aug 23 14:28:01.870241982 (357h1m56.301322123s) [noise=true , address=79.127.205.234:4596 , piece_id=VWEQWYDCV36AHJAQBB7IIH2GNU6UGOQCKOYMKQQH5M3BF5ZAMWXQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.293480028 (357h1m56.724560189s) [noise=true , address=79.127.219.48:4091 , piece_id=ZZN4NXJIRQRSKZZRXIZN4A4ZY7XAZU4WOZQ2773BFZU6R5GWQ5QA, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.312877683 (357h1m56.743957814s) [noise=true , address=79.127.219.51:4212 , piece_id=I4A5G4NB5LZGZ4AKS4ZJXAX6JWFX7I4XDTZGZQ7BTJVGTWHQWQTQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.329870768 (357h1m56.760950899s) [noise=true , address=79.127.219.47:4063 , piece_id=AJHVIZ4TASKYYK525R6KCJU7JEFAVPL6RQTJX53DSQEDZBH5GBFQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.487300324 (357h1m56.918380465s) [noise=true , address=77.90.31.19:4857 , piece_id=KOGKADKRYREAULHF54Y67DYM4V3L7V4AL6EHDVBDWMHYJNKYMLSA, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.489312033 (357h1m56.920392174s) [noise=true , address=77.90.31.13:4649 , piece_id=RFC7SITGS37TZH7LQWMQ7IYBOLARTCSKF6HSLDX5XGJ7OXL7PPEQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.515049289 (357h1m56.94612942s) [noise=true , address=77.90.31.18:4815 , piece_id=JMB6SIQUN4DN3R23NMJMPIRKLCYVJFOK7LUO5KYYNF3FVO7QAWLA, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.516043412 (357h1m56.947123543s) [noise=true , address=77.90.31.11:4567 , piece_id=GDL3HJZW2BZ3Y3LUI2SNWJGBBJEBVRO4XQBJWDUE75JZUEDGP32Q, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.533828992 (357h1m56.964909123s) [noise=true , address=77.90.31.12:4624 , piece_id=H6PI6BL7LMSXNBLA3RR4UVNRU5KHOJXC3SBBJ3NBBZDSJQQ5NF2Q, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.535536099 (357h1m56.966616231s) [noise=true , address=104.36.82.44:4891 , piece_id=FLVR7XEMS2ZEIDSX4XNYEXMCLXXV63KVFOLVJRQVKZLN2LWUWQXQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.599503083 (357h1m57.030583224s) [noise=true , address=104.36.82.20:4461 , piece_id=NGNQCJQKJD32YX5GWV2VHJCW6XMMCGMUW3OYBC5OQSHVTYFQE6WA, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.608928979 (357h1m57.04000911s) [noise=true , address=104.36.82.5:4861 , piece_id=TLUVB7AWEOWHPVNRQGIQ3V6GDSL4AIIEVVEXCMBLJIHIW3HYUTOQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.646317839 (357h1m57.07739797s) [noise=true , address=104.36.82.19:4417 , piece_id=ZQIMDN66BBNEVH7FHX6PX2R4CZPULEWJDLOOZIQXAH7P2ABJSIDQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.650706326 (357h1m57.081786457s) [noise=true , address=77.90.31.10:4543 , piece_id=OPTZPP5SMR7SOL74XHVJJFPZPLTAJCAUEJWX3SZYZJZ2UAIMYPPQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.650730713 (357h1m57.081810844s) [seg_pos={0 0}, upload=371367]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:28:02.650849991 (357h1m57.081930122s) [noise=true , address=104.36.82.52:4119 , piece_id=RHS5Q42HARU5X4NAEPGDEUGYSS64LUARZIVAAXIU3BD2AJSUZ3TQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF
Aug 23 14:28:02.650938300 (357h1m57.082018432s) [noise=true , address=77.90.31.16:4767 , piece_id=4FI74HXN3LRRJ467GI3RVQSJOBPXIJFYZOT3KSX5KO2ZIMK63O6A, seg_pos={0 0}, upload=371367]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF
Aug 23 14:28:02.651003755 (357h1m57.082083886s) [noise=true , address=104.36.82.60:4344 , piece_id=XPVIM4N4OQJTGECEY7G7CDBKK6GXUAR5W7B32MQXHY57RDGT5O5Q, seg_pos={0 0}, upload=371367]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: expected piece hash; context canceled; EOF
Aug 23 14:28:02.651025006 (357h1m57.082105137s) [noise=true , address=79.127.201.215:4343 , piece_id=JNDYG6ZMDU5U6LEF5UBJCGAWVATWRYNELDQAZG35UQ3XTXF5ZSQA, seg_pos={0 0}, upload=371367]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: expected piece hash; context canceled; EOF
Aug 23 14:28:02.651084861 (357h1m57.082165002s) [seg_pos={0 0}, upload=371367]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:28:02.651197546 (357h1m57.082277677s) [upload=371367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:28:02.651209919 (357h1m57.08229006s) [upload=371367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:28:02.994197944 (357h1m57.425278075s) [upload=371367]: Done uploading segments...
Aug 23 14:28:02.994238031 (357h1m57.425318162s) [upload=371367]: Upload finished. err: <nil>
Duration: 6.397065402s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
7.282441366s
FAIL (2025-08-23 14:44:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Aug 23 14:06:34.087047198 (356h40m28.518127319s) [upload=371002]: Starting upload
Aug 23 14:06:34.087103405 (356h40m28.518183536s) [upload=371002]: Done starting upload
Aug 23 14:06:34.087217894 (356h40m28.518298075s) [upload=371002]: Uploading segments...
Aug 23 14:06:34.087280084 (356h40m28.518360225s) [upload=371002]: Got next segment. Inline: false
Aug 23 14:06:34.087333556 (356h40m28.518413697s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:06:34.087345218 (356h40m28.518425379s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:06:34.329714999 (356h40m28.76079513s) [seg_pos={0 0}, upload=371002]: Begin upload segment...
Aug 23 14:06:34.329766266 (356h40m28.760846387s) [seg_pos={0 0}, upload=371002]: Done begin upload segment.
Aug 23 14:06:34.329787617 (356h40m28.760867758s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.329825380 (356h40m28.760905521s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330128550 (356h40m28.761208681s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330176731 (356h40m28.761256862s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330343832 (356h40m28.761423953s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330470363 (356h40m28.761550494s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.330760569 (356h40m28.76184071s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331052808 (356h40m28.762132939s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331345137 (356h40m28.762425268s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.331803835 (356h40m28.762883966s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332093870 (356h40m28.763174001s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332431015 (356h40m28.763511146s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.332721310 (356h40m28.763801441s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333000254 (356h40m28.764080385s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333290029 (356h40m28.76437016s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:34.333579212 (356h40m28.764659343s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Uploading piece...
Aug 23 14:06:36.088000547 (356h40m30.519080668s) [upload=371002]: Next returned nil segment
Aug 23 14:06:36.088019162 (356h40m30.519099293s) [upload=371002]: Waiting for error group managing segments...
Aug 23 14:06:36.365114618 (356h40m30.79619475s) [noise=true , address=45.91.116.14:4191 , piece_id=CUXTPWINB77SCDJYR32IGBZXRYBAL6IBYRYJCQ75A24FQ5Q25VTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.384930313 (356h40m30.816010444s) [noise=true , address=104.36.82.2:4731 , piece_id=WLDPJ6V64VYLTZLVVA3ZKI3O562QTCIGXYFA4IBORG5CHR5TOZOQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.441019685 (356h40m30.872099816s) [noise=true , address=104.36.82.14:4223 , piece_id=KH4AXMHYDGUDXCOH4LIJYK5GSBIP7DWPCUXGCGWA7NN4P7XNVDTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.503003134 (356h40m30.934083265s) [noise=true , address=104.36.82.11:4114 , piece_id=LONWZY5NMDG7KNB4F2LIZRI7ZR44VWLYTFQ4J3WJALCNA7G7BKTA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.773706448 (356h40m31.204786629s) [noise=true , address=77.90.31.21:4878 , piece_id=53UGB5A4JZUBRK5C6TGNEWBNCZKYNPIUC75IGVZPTKBAISLM7IVQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.794301936 (356h40m31.225382068s) [noise=true , address=45.91.116.9:4263 , piece_id=JFZ5X4XI63NOGCWOLH5W4V6JAIPYBVCMHQKOQJVIBHMYCASXS7VA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.797422086 (356h40m31.228502217s) [noise=true , address=45.91.116.5:4299 , piece_id=AU3SJOB4ZCNOLMT3W7ZLTU3EWYRDD4R7TCVJNYTTW25DXGHH6BQA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.891493522 (356h40m31.322573653s) [noise=true , address=45.91.116.6:4975 , piece_id=X7XTECC2ND2IM4645JQZWPP7U54H4ONFTNBR35MYLI2IQP6M2EBQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.893130365 (356h40m31.324210496s) [noise=true , address=104.36.82.4:4804 , piece_id=5MVRMKZHX6DILFOFBTH45DGML7KEXXOL7IOP7LUJSANLD6PHJ72Q, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.926291877 (356h40m31.357372008s) [noise=true , address=77.90.31.19:4843 , piece_id=3DS3QVAFSUYPUYUJPUG3VRYYE6AGRAWD37ZBBTIYBRZRZL64YN5A, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.980332537 (356h40m31.411412668s) [noise=true , address=45.91.116.11:4118 , piece_id=PGVMJNQZI7FBKT7L3D4PFB7OFBLR4DDLSLYMUGVFK6O5DFXO3JKA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:36.986328430 (356h40m31.417408551s) [noise=true , address=77.90.31.15:4697 , piece_id=TOMBZDKSCCRO7LCDXVKJCRJBNDEWSFUR347EK4TRDK7JDHL6SQDA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126158031 (356h40m31.557238152s) [noise=true , address=77.90.31.12:4592 , piece_id=DOVGJ5E2MLWE4UY6CTTNV2ASJDAG3EVKYMO6PI2KJRCMTFHJ5QSQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: <nil>
Aug 23 14:06:37.126185253 (356h40m31.557265384s) [seg_pos={0 0}, upload=371002]: Segment reached optimal threshold of 13 pieces.
Aug 23 14:06:37.126290142 (356h40m31.557370274s) [noise=true , address=104.36.82.9:4005 , piece_id=4LZPJYP2VGA6DWIIVEYWMBENW23FTQRA67F3UOUP26UJ3LLMALZQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126445190 (356h40m31.557525321s) [noise=true , address=104.36.82.12:4151 , piece_id=HNJSSBJEPFUI2U5HGVZYOHLJLONQQTJZUMKNYJMM7SMIN6TUDWTQ, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58508->104.36.82.12:4151: use of closed network connection
Aug 23 14:06:37.126492731 (356h40m31.557572862s) [noise=true , address=77.90.31.10:4519 , piece_id=SPDBZ7KCUKEFYLQHKY2FHRKGITRCMCCTBECWQ7L4AIC3ZXLYSMZA, seg_pos={0 0}, upload=371002]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF
Aug 23 14:06:37.126560811 (356h40m31.557640952s) [seg_pos={0 0}, upload=371002]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 23 14:06:37.126686101 (356h40m31.557766232s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:06:37.126693204 (356h40m31.557773365s) [upload=371002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:06:37.513830133 (356h40m31.944910264s) [upload=371002]: Done uploading segments...
Aug 23 14:06:37.513848849 (356h40m31.944928981s) [upload=371002]: Upload finished. err: <nil>
Duration: 7.282466796s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............................X.......................X....] duration:
1.893363089s
FAIL (2025-08-23 14:44:47 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.893391194s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................................X.......................X...] duration:
1.948481956s
FAIL (2025-08-23 14:44:19 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 23 14:42:29.380030301 (357h16m23.811110432s) [upload=371618]: Starting upload Aug 23 14:42:29.380079075 (357h16m23.811159206s) [upload=371618]: Done starting upload Aug 23 14:42:29.380138919 (357h16m23.81121905s) [upload=371618]: Uploading segments... Aug 23 14:42:29.380179878 (357h16m23.811259999s) [upload=371618]: Got next segment. Inline: true Aug 23 14:42:29.380183835 (357h16m23.811263966s) [upload=371618]: Waiting for error group managing segments... Aug 23 14:42:29.380215255 (357h16m23.811295386s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 23 14:42:29.380219483 (357h16m23.811299614s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 23 14:42:29.380221898 (357h16m23.811302029s) [upload=371618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 23 14:42:29.595805998 (357h16m24.026886129s) [upload=371618]: Done uploading segments... Aug 23 14:42:29.595824122 (357h16m24.026904253s) [upload=371618]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.948506814s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [....X.........X..................X....X.X...................] duration:
6.337520588s
FAIL (2025-08-23 14:44:15 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: invalid range 0:4194304 (size:4194304)
======LOGS======
Aug 23 14:27:59.916879691 (357h1m54.347959822s) [upload=371367]: Starting upload
Aug 23 14:27:59.916927211 (357h1m54.348007342s) [upload=371367]: Done starting upload
Aug 23 14:27:59.916964042 (357h1m54.348044183s) [upload=371367]: Uploading segments...
Aug 23 14:27:59.917143525 (357h1m54.348223656s) [upload=371367]: Got next segment. Inline: false
Aug 23 14:27:59.917178773 (357h1m54.348258894s) [upload=371367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 14:27:59.917183542 (357h1m54.348263663s) [upload=371367]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 14:28:00.381688206 (357h1m54.812768337s) [seg_pos={0 0}, upload=371367]: Begin upload segment...
Aug 23 14:28:00.381881216 (357h1m54.812961357s) [seg_pos={0 0}, upload=371367]: Done begin upload segment.
Aug 23 14:28:00.381921983 (357h1m54.813002114s) [noise=true , address=79.127.205.234:4596 , piece_id=VWEQWYDCV36AHJAQBB7IIH2GNU6UGOQCKOYMKQQH5M3BF5ZAMWXQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.381955859 (357h1m54.813036s) [noise=true , address=104.36.82.60:4344 , piece_id=XPVIM4N4OQJTGECEY7G7CDBKK6GXUAR5W7B32MQXHY57RDGT5O5Q, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.382447860 (357h1m54.813527991s) [noise=true , address=104.36.82.44:4891 , piece_id=FLVR7XEMS2ZEIDSX4XNYEXMCLXXV63KVFOLVJRQVKZLN2LWUWQXQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.382500691 (357h1m54.813580822s) [noise=true , address=79.127.201.215:4343 , piece_id=JNDYG6ZMDU5U6LEF5UBJCGAWVATWRYNELDQAZG35UQ3XTXF5ZSQA, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.382775156 (357h1m54.813855288s) [noise=true , address=77.90.31.11:4567 , piece_id=GDL3HJZW2BZ3Y3LUI2SNWJGBBJEBVRO4XQBJWDUE75JZUEDGP32Q, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.382817186 (357h1m54.813897317s) [noise=true , address=77.90.31.13:4649 , piece_id=RFC7SITGS37TZH7LQWMQ7IYBOLARTCSKF6HSLDX5XGJ7OXL7PPEQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.383157026 (357h1m54.814237157s) [noise=true , address=77.90.31.16:4767 , piece_id=4FI74HXN3LRRJ467GI3RVQSJOBPXIJFYZOT3KSX5KO2ZIMK63O6A, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.383182946 (357h1m54.814263077s) [noise=true , address=77.90.31.18:4815 , piece_id=JMB6SIQUN4DN3R23NMJMPIRKLCYVJFOK7LUO5KYYNF3FVO7QAWLA, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.383562633 (357h1m54.814642774s) [noise=true , address=77.90.31.10:4543 , piece_id=OPTZPP5SMR7SOL74XHVJJFPZPLTAJCAUEJWX3SZYZJZ2UAIMYPPQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.383910348 (357h1m54.814990479s) [noise=true , address=79.127.219.47:4063 , piece_id=AJHVIZ4TASKYYK525R6KCJU7JEFAVPL6RQTJX53DSQEDZBH5GBFQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.384252182 (357h1m54.815332313s) [noise=true , address=79.127.219.48:4091 , piece_id=ZZN4NXJIRQRSKZZRXIZN4A4ZY7XAZU4WOZQ2773BFZU6R5GWQ5QA, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.384635074 (357h1m54.815715205s) [noise=true , address=104.36.82.19:4417 , piece_id=ZQIMDN66BBNEVH7FHX6PX2R4CZPULEWJDLOOZIQXAH7P2ABJSIDQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.384965086 (357h1m54.816045217s) [noise=true , address=104.36.82.52:4119 , piece_id=RHS5Q42HARU5X4NAEPGDEUGYSS64LUARZIVAAXIU3BD2AJSUZ3TQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.385316959 (357h1m54.81639709s) [noise=true , address=77.90.31.19:4857 , piece_id=KOGKADKRYREAULHF54Y67DYM4V3L7V4AL6EHDVBDWMHYJNKYMLSA, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.385618246 (357h1m54.816698387s) [noise=true , address=79.127.219.51:4212 , piece_id=I4A5G4NB5LZGZ4AKS4ZJXAX6JWFX7I4XDTZGZQ7BTJVGTWHQWQTQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.385957936 (357h1m54.817038067s) [noise=true , address=77.90.31.12:4624 , piece_id=H6PI6BL7LMSXNBLA3RR4UVNRU5KHOJXC3SBBJ3NBBZDSJQQ5NF2Q, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.386250415 (357h1m54.817330566s) [noise=true , address=104.36.82.20:4461 , piece_id=NGNQCJQKJD32YX5GWV2VHJCW6XMMCGMUW3OYBC5OQSHVTYFQE6WA, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:00.386596428 (357h1m54.817676569s) [noise=true , address=104.36.82.5:4861 , piece_id=TLUVB7AWEOWHPVNRQGIQ3V6GDSL4AIIEVVEXCMBLJIHIW3HYUTOQ, seg_pos={0 0}, upload=371367]: Uploading piece...
Aug 23 14:28:01.551634527 (357h1m55.982714658s) [upload=371367]: Next returned nil segment
Aug 23 14:28:01.551654415 (357h1m55.982734556s) [upload=371367]: Waiting for error group managing segments...
Aug 23 14:28:01.870241982 (357h1m56.301322123s) [noise=true , address=79.127.205.234:4596 , piece_id=VWEQWYDCV36AHJAQBB7IIH2GNU6UGOQCKOYMKQQH5M3BF5ZAMWXQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.293480028 (357h1m56.724560189s) [noise=true , address=79.127.219.48:4091 , piece_id=ZZN4NXJIRQRSKZZRXIZN4A4ZY7XAZU4WOZQ2773BFZU6R5GWQ5QA, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.312877683 (357h1m56.743957814s) [noise=true , address=79.127.219.51:4212 , piece_id=I4A5G4NB5LZGZ4AKS4ZJXAX6JWFX7I4XDTZGZQ7BTJVGTWHQWQTQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.329870768 (357h1m56.760950899s) [noise=true , address=79.127.219.47:4063 , piece_id=AJHVIZ4TASKYYK525R6KCJU7JEFAVPL6RQTJX53DSQEDZBH5GBFQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.487300324 (357h1m56.918380465s) [noise=true , address=77.90.31.19:4857 , piece_id=KOGKADKRYREAULHF54Y67DYM4V3L7V4AL6EHDVBDWMHYJNKYMLSA, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.489312033 (357h1m56.920392174s) [noise=true , address=77.90.31.13:4649 , piece_id=RFC7SITGS37TZH7LQWMQ7IYBOLARTCSKF6HSLDX5XGJ7OXL7PPEQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.515049289 (357h1m56.94612942s) [noise=true , address=77.90.31.18:4815 , piece_id=JMB6SIQUN4DN3R23NMJMPIRKLCYVJFOK7LUO5KYYNF3FVO7QAWLA, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.516043412 (357h1m56.947123543s) [noise=true , address=77.90.31.11:4567 , piece_id=GDL3HJZW2BZ3Y3LUI2SNWJGBBJEBVRO4XQBJWDUE75JZUEDGP32Q, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.533828992 (357h1m56.964909123s) [noise=true , address=77.90.31.12:4624 , piece_id=H6PI6BL7LMSXNBLA3RR4UVNRU5KHOJXC3SBBJ3NBBZDSJQQ5NF2Q, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.535536099 (357h1m56.966616231s) [noise=true , address=104.36.82.44:4891 , piece_id=FLVR7XEMS2ZEIDSX4XNYEXMCLXXV63KVFOLVJRQVKZLN2LWUWQXQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.599503083 (357h1m57.030583224s) [noise=true , address=104.36.82.20:4461 , piece_id=NGNQCJQKJD32YX5GWV2VHJCW6XMMCGMUW3OYBC5OQSHVTYFQE6WA, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.608928979 (357h1m57.04000911s) [noise=true , address=104.36.82.5:4861 , piece_id=TLUVB7AWEOWHPVNRQGIQ3V6GDSL4AIIEVVEXCMBLJIHIW3HYUTOQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.646317839 (357h1m57.07739797s) [noise=true , address=104.36.82.19:4417 , piece_id=ZQIMDN66BBNEVH7FHX6PX2R4CZPULEWJDLOOZIQXAH7P2ABJSIDQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.650706326 (357h1m57.081786457s) [noise=true , address=77.90.31.10:4543 , piece_id=OPTZPP5SMR7SOL74XHVJJFPZPLTAJCAUEJWX3SZYZJZ2UAIMYPPQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: <nil>
Aug 23 14:28:02.650730713 (357h1m57.081810844s) [seg_pos={0 0}, upload=371367]: Segment reached optimal threshold of 14 pieces.
Aug 23 14:28:02.650849991 (357h1m57.081930122s) [noise=true , address=104.36.82.52:4119 , piece_id=RHS5Q42HARU5X4NAEPGDEUGYSS64LUARZIVAAXIU3BD2AJSUZ3TQ, seg_pos={0 0}, upload=371367]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF
Aug 23 14:28:02.650938300 (357h1m57.082018432s) [noise=true , address=77.90.31.16:4767 , piece_id=4FI74HXN3LRRJ467GI3RVQSJOBPXIJFYZOT3KSX5KO2ZIMK63O6A, seg_pos={0 0}, upload=371367]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF
Aug 23 14:28:02.651003755 (357h1m57.082083886s) [noise=true , address=104.36.82.60:4344 , piece_id=XPVIM4N4OQJTGECEY7G7CDBKK6GXUAR5W7B32MQXHY57RDGT5O5Q, seg_pos={0 0}, upload=371367]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: expected piece hash; context canceled; EOF
Aug 23 14:28:02.651025006 (357h1m57.082105137s) [noise=true , address=79.127.201.215:4343 , piece_id=JNDYG6ZMDU5U6LEF5UBJCGAWVATWRYNELDQAZG35UQ3XTXF5ZSQA, seg_pos={0 0}, upload=371367]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: expected piece hash; context canceled; EOF
Aug 23 14:28:02.651084861 (357h1m57.082165002s) [seg_pos={0 0}, upload=371367]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 23 14:28:02.651197546 (357h1m57.082277677s) [upload=371367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 23 14:28:02.651209919 (357h1m57.08229006s) [upload=371367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 14:28:02.994197944 (357h1m57.425278075s) [upload=371367]: Done uploading segments...
Aug 23 14:28:02.994238031 (357h1m57.425318162s) [upload=371367]: Upload finished. err: <nil>
Duration: 6.337545126s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|