Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X.........X..........] duration:
1.852538938s
FAIL (2025-08-07 12:26:09 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.852563424s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........................................X.........X.........] duration:
1.954086285s
FAIL (2025-08-07 12:25:33 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.954112825s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........................................X.........X........] duration:
1.831307377s
FAIL (2025-08-07 12:25:06 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.831365919s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.66769077s
FAIL (2025-08-07 12:24:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.667723351s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........................................X.........X.......] duration:
2.147701761s
FAIL (2025-08-07 12:24:37 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.14773298s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.027190729s
FAIL (2025-08-07 12:23:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.027216328s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...........................................X.........X......] duration:
1.841367083s
FAIL (2025-08-07 12:23:25 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.841394655s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [............................................X.........X.....] duration:
1.847086233s
FAIL (2025-08-07 12:23:01 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.847136177s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
5.866173761s
FAIL (2025-08-07 12:22:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.866201373s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............................................X.........X....] duration:
1.849845352s
FAIL (2025-08-07 12:22:08 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.849868716s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
5.828980999s
FAIL (2025-08-07 12:21:52 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.82900875s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............................................X.........X...] duration:
1.85509179s
FAIL (2025-08-07 12:21:14 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.855157845s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
6.003866901s
FAIL (2025-08-07 12:21:06 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.00389226s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............................................X.........X..] duration:
2.042849873s
FAIL (2025-08-07 12:20:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.042875099s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
5.916902445s
FAIL (2025-08-07 12:20:23 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.916934536s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [X...............................................X.........X.] duration:
1.879641446s
FAIL (2025-08-07 12:19:29 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.879699146s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
5.721174394s
FAIL (2025-08-07 12:19:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.721197096s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.X...............................................X.........X] duration:
862.158498ms
FAIL (2025-08-07 12:18:42 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== keInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil> Aug 7 12:13:41.982256263 (2h33m41.57160125s) [upload=2684]: Starting upload Aug 7 12:13:41.982312901 (2h33m41.571657888s) [upload=2684]: Done starting upload Aug 7 12:13:41.982380600 (2h33m41.571725578s) [upload=2684]: Uploading segments... Aug 7 12:13:41.982410558 (2h33m41.571755535s) [upload=2684]: Got next segment. Inline: true Aug 7 12:13:41.982414355 (2h33m41.571759342s) [upload=2684]: Waiting for error group managing segments... Aug 7 12:13:41.982456606 (2h33m41.571801583s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:41.982460323 (2h33m41.57180531s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:41.982462958 (2h33m41.571807945s) [upload=2684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:42.316448451 (2h33m41.905793438s) [upload=2684]: Done uploading segments... Aug 7 12:13:42.316484750 (2h33m41.905829737s) [upload=2684]: Upload finished. err: <nil> Aug 7 12:14:27.416938530 (2h34m27.006283517s) [upload=2697]: Starting upload Aug 7 12:14:27.416985209 (2h34m27.006330186s) [upload=2697]: Done starting upload Aug 7 12:14:27.417038621 (2h34m27.006383608s) [upload=2697]: Uploading segments... Aug 7 12:14:27.417071604 (2h34m27.006416591s) [upload=2697]: Got next segment. Inline: true Aug 7 12:14:27.417075593 (2h34m27.00642058s) [upload=2697]: Waiting for error group managing segments... Aug 7 12:14:27.417107674 (2h34m27.006452661s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:27.417111571 (2h34m27.006456558s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:27.417113785 (2h34m27.006458762s) [upload=2697]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:27.712529751 (2h34m27.301874738s) [upload=2697]: Done uploading segments... Aug 7 12:14:27.712548447 (2h34m27.301893434s) [upload=2697]: Upload finished. err: <nil> Aug 7 12:14:52.046663563 (2h34m51.63600855s) [upload=2705]: Starting upload Aug 7 12:14:52.046707037 (2h34m51.636052014s) [upload=2705]: Done starting upload Aug 7 12:14:52.046907530 (2h34m51.636252517s) [upload=2705]: Uploading segments... Aug 7 12:14:52.046960011 (2h34m51.636304988s) [upload=2705]: Got next segment. Inline: true Aug 7 12:14:52.046976773 (2h34m51.63632176s) [upload=2705]: Waiting for error group managing segments... Aug 7 12:14:52.047015928 (2h34m51.636360915s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:14:52.047021709 (2h34m51.636366696s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:14:52.047023913 (2h34m51.6363689s) [upload=2705]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:14:52.427197045 (2h34m52.016542092s) [upload=2705]: Done uploading segments... Aug 7 12:14:52.427220089 (2h34m52.016565076s) [upload=2705]: Upload finished. err: <nil> Aug 7 12:15:40.832326180 (2h35m40.421671167s) [upload=2718]: Starting upload Aug 7 12:15:40.832371927 (2h35m40.421716914s) [upload=2718]: Done starting upload Aug 7 12:15:40.832449926 (2h35m40.421794913s) [upload=2718]: Uploading segments... Aug 7 12:15:40.832567562 (2h35m40.421912549s) [upload=2718]: Got next segment. Inline: true Aug 7 12:15:40.832572641 (2h35m40.421917618s) [upload=2718]: Waiting for error group managing segments... Aug 7 12:15:40.832619831 (2h35m40.421964818s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:15:40.832625753 (2h35m40.42197074s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:15:40.832628709 (2h35m40.421973696s) [upload=2718]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:15:41.159949940 (2h35m40.749294927s) [upload=2718]: Done uploading segments... Aug 7 12:15:41.159969467 (2h35m40.749314464s) [upload=2718]: Upload finished. err: <nil> Aug 7 12:16:12.558606782 (2h36m12.147951769s) [upload=2727]: Starting upload Aug 7 12:16:12.558657569 (2h36m12.148002556s) [upload=2727]: Done starting upload Aug 7 12:16:12.558712294 (2h36m12.148057281s) [upload=2727]: Uploading segments... Aug 7 12:16:12.558788809 (2h36m12.148133796s) [upload=2727]: Got next segment. Inline: true Aug 7 12:16:12.558793318 (2h36m12.148138305s) [upload=2727]: Waiting for error group managing segments... Aug 7 12:16:12.558829889 (2h36m12.148174876s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:12.558833856 (2h36m12.148178843s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:12.558836401 (2h36m12.148181388s) [upload=2727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:12.845368413 (2h36m12.4347134s) [upload=2727]: Done uploading segments... Aug 7 12:16:12.845389223 (2h36m12.4347342s) [upload=2727]: Upload finished. err: <nil> Aug 7 12:16:28.193778419 (2h36m27.783123406s) [upload=2730]: Starting upload Aug 7 12:16:28.193846841 (2h36m27.783191828s) [upload=2730]: Done starting upload Aug 7 12:16:28.193911424 (2h36m27.783256411s) [upload=2730]: Uploading segments... Aug 7 12:16:28.193946051 (2h36m27.783291038s) [upload=2730]: Got next segment. Inline: true Aug 7 12:16:28.193950078 (2h36m27.783295065s) [upload=2730]: Waiting for error group managing segments... Aug 7 12:16:28.193998620 (2h36m27.783343608s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:28.194002930 (2h36m27.783347917s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:28.194005735 (2h36m27.783350722s) [upload=2730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:28.476261665 (2h36m28.065606652s) [upload=2730]: Done uploading segments... Aug 7 12:16:28.476288386 (2h36m28.065633393s) [upload=2730]: Upload finished. err: <nil> Aug 7 12:16:48.229313899 (2h36m47.818658886s) [upload=2739]: Starting upload Aug 7 12:16:48.229366840 (2h36m47.818711827s) [upload=2739]: Done starting upload Aug 7 12:16:48.229417327 (2h36m47.818762304s) [upload=2739]: Uploading segments... Aug 7 12:16:48.229446482 (2h36m47.818791469s) [upload=2739]: Got next segment. Inline: true Aug 7 12:16:48.229450540 (2h36m47.818795527s) [upload=2739]: Waiting for error group managing segments... Aug 7 12:16:48.229485948 (2h36m47.818830936s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:16:48.229489866 (2h36m47.818834853s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:16:48.229492250 (2h36m47.818837227s) [upload=2739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:16:48.513365526 (2h36m48.102710513s) [upload=2739]: Done uploading segments... Aug 7 12:16:48.513383730 (2h36m48.102728717s) [upload=2739]: Upload finished. err: <nil> Aug 7 12:17:52.179425653 (2h37m51.76877064s) [upload=2756]: Starting upload Aug 7 12:17:52.179519132 (2h37m51.768864119s) [upload=2756]: Done starting upload Aug 7 12:17:52.179571593 (2h37m51.76891661s) [upload=2756]: Uploading segments... Aug 7 12:17:52.179605036 (2h37m51.768950023s) [upload=2756]: Got next segment. Inline: true Aug 7 12:17:52.179608753 (2h37m51.76895374s) [upload=2756]: Waiting for error group managing segments... Aug 7 12:17:52.179644451 (2h37m51.768989438s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:17:52.179648480 (2h37m51.768993467s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:17:52.179650914 (2h37m51.768995901s) [upload=2756]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:17:52.534478653 (2h37m52.12382363s) [upload=2756]: Done uploading segments... Aug 7 12:17:52.534493310 (2h37m52.123838297s) [upload=2756]: Upload finished. err: <nil> Aug 7 12:18:09.499452972 (2h38m9.088797949s) [upload=2762]: Starting upload Aug 7 12:18:09.499528506 (2h38m9.088873493s) [upload=2762]: Done starting upload Aug 7 12:18:09.499587800 (2h38m9.088932787s) [upload=2762]: Uploading segments... Aug 7 12:18:09.499619700 (2h38m9.088964687s) [upload=2762]: Got next segment. Inline: true Aug 7 12:18:09.499623508 (2h38m9.088968495s) [upload=2762]: Waiting for error group managing segments... Aug 7 12:18:09.499654727 (2h38m9.088999714s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:09.499658444 (2h38m9.089003431s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:09.499660709 (2h38m9.089005696s) [upload=2762]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:09.864969942 (2h38m9.454314929s) [upload=2762]: Done uploading segments... Aug 7 12:18:09.864987755 (2h38m9.454332742s) [upload=2762]: Upload finished. err: <nil> Aug 7 12:18:42.437345297 (2h38m42.026690285s) [upload=2773]: Starting upload Aug 7 12:18:42.437393620 (2h38m42.026738597s) [upload=2773]: Done starting upload Aug 7 12:18:42.437446101 (2h38m42.026791088s) [upload=2773]: Uploading segments... Aug 7 12:18:42.437478072 (2h38m42.026823059s) [upload=2773]: Got next segment. Inline: true Aug 7 12:18:42.437483512 (2h38m42.026828499s) [upload=2773]: Waiting for error group managing segments... Aug 7 12:18:42.437514521 (2h38m42.026859508s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:18:42.437518278 (2h38m42.026863265s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:18:42.437520512 (2h38m42.026865499s) [upload=2773]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:18:42.650164638 (2h38m42.239509625s) [upload=2773]: Done uploading segments... Aug 7 12:18:42.650181309 (2h38m42.239526296s) [upload=2773]: Upload finished. err: metaclient: Too Many Requests
Duration: 862.179769ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
5.602567088s
FAIL (2025-08-07 12:18:16 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.602591255s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..X...............................................X.........] duration:
2.027228755s
FAIL (2025-08-07 12:18:10 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after df11e3eae1ac8f408d918f55d929d54d4e5df3d8eaae7e4f8e4c7c8038d1c94c != before bd9ccba63d0a10a25cb9e58291141281665e96854a4357fa64c5bb1ae08d3d6f ======LOGS====== *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:06:53.059666435 (2h26m52.649011472s) [upload=2569]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:06:53.386093536 (2h26m52.975438523s) [upload=2569]: Done uploading segments... Aug 7 12:06:53.386195922 (2h26m52.975540909s) [upload=2569]: Upload finished. err: <nil> Aug 7 12:07:12.996984187 (2h27m12.586329175s) [upload=2577]: Starting upload Aug 7 12:07:12.997029885 (2h27m12.586374872s) [upload=2577]: Done starting upload Aug 7 12:07:12.997091613 (2h27m12.5864366s) [upload=2577]: Uploading segments... Aug 7 12:07:12.997174532 (2h27m12.586519509s) [upload=2577]: Got next segment. Inline: true Aug 7 12:07:12.997179731 (2h27m12.586524719s) [upload=2577]: Waiting for error group managing segments... Aug 7 12:07:12.997230749 (2h27m12.586575726s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:12.997236009 (2h27m12.586580996s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:12.997238444 (2h27m12.586583431s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:13.277508374 (2h27m12.866853361s) [upload=2577]: Done uploading segments... Aug 7 12:07:13.277529685 (2h27m12.866874672s) [upload=2577]: Upload finished. err: <nil> Aug 7 12:07:59.647444303 (2h27m59.23678929s) [upload=2586]: Starting upload Aug 7 12:07:59.647485382 (2h27m59.236830369s) [upload=2586]: Done starting upload Aug 7 12:07:59.647544936 (2h27m59.236889923s) [upload=2586]: Uploading segments... Aug 7 12:07:59.647582257 (2h27m59.236927234s) [upload=2586]: Got next segment. Inline: true Aug 7 12:07:59.647585894 (2h27m59.236930881s) [upload=2586]: Waiting for error group managing segments... Aug 7 12:07:59.647649967 (2h27m59.236994954s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:59.647654155 (2h27m59.236999142s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:59.647656389 (2h27m59.237001376s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:59.961215120 (2h27m59.550560107s) [upload=2586]: Done uploading segments... Aug 7 12:07:59.961233604 (2h27m59.550578581s) [upload=2586]: Upload finished. err: <nil> Aug 7 12:09:04.474008716 (2h29m4.063353693s) [upload=2606]: Starting upload Aug 7 12:09:04.474054282 (2h29m4.063399269s) [upload=2606]: Done starting upload Aug 7 12:09:04.474117544 (2h29m4.063462531s) [upload=2606]: Uploading segments... Aug 7 12:09:04.474146590 (2h29m4.063491567s) [upload=2606]: Got next segment. Inline: true Aug 7 12:09:04.474149866 (2h29m4.063494853s) [upload=2606]: Waiting for error group managing segments... Aug 7 12:09:04.474185023 (2h29m4.06353001s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:04.474190423 (2h29m4.0635354s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:04.474193168 (2h29m4.063538145s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:04.800005844 (2h29m4.389350831s) [upload=2606]: Done uploading segments... Aug 7 12:09:04.800024600 (2h29m4.389369587s) [upload=2606]: Upload finished. err: <nil> Aug 7 12:09:43.798285096 (2h29m43.387630083s) [upload=2616]: Starting upload Aug 7 12:09:43.798345140 (2h29m43.387690127s) [upload=2616]: Done starting upload Aug 7 12:09:43.798408411 (2h29m43.387753398s) [upload=2616]: Uploading segments... Aug 7 12:09:43.798439370 (2h29m43.387784357s) [upload=2616]: Got next segment. Inline: true Aug 7 12:09:43.798443217 (2h29m43.387788204s) [upload=2616]: Waiting for error group managing segments... Aug 7 12:09:43.798475981 (2h29m43.387820968s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:43.798479978 (2h29m43.387824965s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:43.798482313 (2h29m43.38782729s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:44.143582671 (2h29m43.732927658s) [upload=2616]: Done uploading segments... Aug 7 12:09:44.143598691 (2h29m43.732943668s) [upload=2616]: Upload finished. err: <nil> Aug 7 12:10:27.584576925 (2h30m27.173921912s) [upload=2629]: Starting upload Aug 7 12:10:27.584629195 (2h30m27.173974172s) [upload=2629]: Done starting upload Aug 7 12:10:27.584678159 (2h30m27.174023146s) [upload=2629]: Uploading segments... Aug 7 12:10:27.584707415 (2h30m27.174052402s) [upload=2629]: Got next segment. Inline: true Aug 7 12:10:27.584711312 (2h30m27.174056299s) [upload=2629]: Waiting for error group managing segments... Aug 7 12:10:27.584743203 (2h30m27.17408819s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:10:27.584747040 (2h30m27.174092027s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:10:27.584749435 (2h30m27.174094422s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:10:27.887810742 (2h30m27.477155729s) [upload=2629]: Done uploading segments... Aug 7 12:10:27.887828937 (2h30m27.477173914s) [upload=2629]: Upload finished. err: <nil> Aug 7 12:11:12.432073435 (2h31m12.021418422s) [upload=2642]: Starting upload Aug 7 12:11:12.432118301 (2h31m12.021463278s) [upload=2642]: Done starting upload Aug 7 12:11:12.432168257 (2h31m12.021513244s) [upload=2642]: Uploading segments... Aug 7 12:11:12.432197673 (2h31m12.02154266s) [upload=2642]: Got next segment. Inline: true Aug 7 12:11:12.432201360 (2h31m12.021546347s) [upload=2642]: Waiting for error group managing segments... Aug 7 12:11:12.432235225 (2h31m12.021580212s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:11:12.432238791 (2h31m12.021583778s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:11:12.432241005 (2h31m12.021585992s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:11:12.826402410 (2h31m12.415747397s) [upload=2642]: Done uploading segments... Aug 7 12:11:12.826420154 (2h31m12.415765141s) [upload=2642]: Upload finished. err: <nil> Aug 7 12:12:01.525617822 (2h32m1.114962819s) [upload=2656]: Starting upload Aug 7 12:12:01.525663079 (2h32m1.115008066s) [upload=2656]: Done starting upload Aug 7 12:12:01.525738163 (2h32m1.11508316s) [upload=2656]: Uploading segments... Aug 7 12:12:01.525854065 (2h32m1.115199052s) [upload=2656]: Got next segment. Inline: true Aug 7 12:12:01.525860697 (2h32m1.115205684s) [upload=2656]: Waiting for error group managing segments... Aug 7 12:12:01.525912907 (2h32m1.115257894s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:01.525919960 (2h32m1.115264957s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:01.525923287 (2h32m1.115268284s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:01.969933727 (2h32m1.559278714s) [upload=2656]: Done uploading segments... Aug 7 12:12:01.969966339 (2h32m1.559311326s) [upload=2656]: Upload finished. err: <nil> Aug 7 12:12:23.526243134 (2h32m23.115588181s) [upload=2664]: Starting upload Aug 7 12:12:23.526339669 (2h32m23.115684696s) [upload=2664]: Done starting upload Aug 7 12:12:23.526447936 (2h32m23.115792923s) [upload=2664]: Uploading segments... Aug 7 12:12:23.526522438 (2h32m23.115867445s) [upload=2664]: Got next segment. Inline: true Aug 7 12:12:23.526530534 (2h32m23.115875551s) [upload=2664]: Waiting for error group managing segments... Aug 7 12:12:23.526585439 (2h32m23.115930446s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:23.526592743 (2h32m23.11593775s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:23.526597251 (2h32m23.115942248s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:23.902417816 (2h32m23.491762803s) [upload=2664]: Done uploading segments... Aug 7 12:12:23.902436842 (2h32m23.491781829s) [upload=2664]: Upload finished. err: <nil> Aug 7 12:12:56.180563006 (2h32m55.769907993s) [upload=2668]: Starting upload Aug 7 12:12:56.180605848 (2h32m55.769950835s) [upload=2668]: Done starting upload Aug 7 12:12:56.180665191 (2h32m55.770010178s) [upload=2668]: Uploading segments... Aug 7 12:12:56.180703014 (2h32m55.770048001s) [upload=2668]: Got next segment. Inline: true Aug 7 12:12:56.180706641 (2h32m55.770051618s) [upload=2668]: Waiting for error group managing segments... Aug 7 12:12:56.180740676 (2h32m55.770085663s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:56.180744884 (2h32m55.770089871s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:56.180747298 (2h32m55.770092285s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:56.490488305 (2h32m56.079833292s) [upload=2668]: Done uploading segments... Aug 7 12:12:56.490511128 (2h32m56.079856115s) [upload=2668]: Upload finished. err: <nil> Aug 7 12:13:11.317339795 (2h33m10.906684772s) [upload=2675]: Starting upload Aug 7 12:13:11.317388448 (2h33m10.906733435s) [upload=2675]: Done starting upload Aug 7 12:13:11.317435919 (2h33m10.906780906s) [upload=2675]: Uploading segments... Aug 7 12:13:11.317477428 (2h33m10.906822415s) [upload=2675]: Got next segment. Inline: true Aug 7 12:13:11.317481295 (2h33m10.906826283s) [upload=2675]: Waiting for error group managing segments... Aug 7 12:13:11.317516183 (2h33m10.90686117s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:11.317520180 (2h33m10.906865168s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil>
Duration: 2.027256558s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...X...............................................X........] duration:
1.939154614s
FAIL (2025-08-07 12:17:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after df11e3eae1ac8f408d918f55d929d54d4e5df3d8eaae7e4f8e4c7c8038d1c94c != before bd9ccba63d0a10a25cb9e58291141281665e96854a4357fa64c5bb1ae08d3d6f ======LOGS====== *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:06:53.059666435 (2h26m52.649011472s) [upload=2569]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:06:53.386093536 (2h26m52.975438523s) [upload=2569]: Done uploading segments... Aug 7 12:06:53.386195922 (2h26m52.975540909s) [upload=2569]: Upload finished. err: <nil> Aug 7 12:07:12.996984187 (2h27m12.586329175s) [upload=2577]: Starting upload Aug 7 12:07:12.997029885 (2h27m12.586374872s) [upload=2577]: Done starting upload Aug 7 12:07:12.997091613 (2h27m12.5864366s) [upload=2577]: Uploading segments... Aug 7 12:07:12.997174532 (2h27m12.586519509s) [upload=2577]: Got next segment. Inline: true Aug 7 12:07:12.997179731 (2h27m12.586524719s) [upload=2577]: Waiting for error group managing segments... Aug 7 12:07:12.997230749 (2h27m12.586575726s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:12.997236009 (2h27m12.586580996s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:12.997238444 (2h27m12.586583431s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:13.277508374 (2h27m12.866853361s) [upload=2577]: Done uploading segments... Aug 7 12:07:13.277529685 (2h27m12.866874672s) [upload=2577]: Upload finished. err: <nil> Aug 7 12:07:59.647444303 (2h27m59.23678929s) [upload=2586]: Starting upload Aug 7 12:07:59.647485382 (2h27m59.236830369s) [upload=2586]: Done starting upload Aug 7 12:07:59.647544936 (2h27m59.236889923s) [upload=2586]: Uploading segments... Aug 7 12:07:59.647582257 (2h27m59.236927234s) [upload=2586]: Got next segment. Inline: true Aug 7 12:07:59.647585894 (2h27m59.236930881s) [upload=2586]: Waiting for error group managing segments... Aug 7 12:07:59.647649967 (2h27m59.236994954s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:59.647654155 (2h27m59.236999142s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:59.647656389 (2h27m59.237001376s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:59.961215120 (2h27m59.550560107s) [upload=2586]: Done uploading segments... Aug 7 12:07:59.961233604 (2h27m59.550578581s) [upload=2586]: Upload finished. err: <nil> Aug 7 12:09:04.474008716 (2h29m4.063353693s) [upload=2606]: Starting upload Aug 7 12:09:04.474054282 (2h29m4.063399269s) [upload=2606]: Done starting upload Aug 7 12:09:04.474117544 (2h29m4.063462531s) [upload=2606]: Uploading segments... Aug 7 12:09:04.474146590 (2h29m4.063491567s) [upload=2606]: Got next segment. Inline: true Aug 7 12:09:04.474149866 (2h29m4.063494853s) [upload=2606]: Waiting for error group managing segments... Aug 7 12:09:04.474185023 (2h29m4.06353001s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:04.474190423 (2h29m4.0635354s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:04.474193168 (2h29m4.063538145s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:04.800005844 (2h29m4.389350831s) [upload=2606]: Done uploading segments... Aug 7 12:09:04.800024600 (2h29m4.389369587s) [upload=2606]: Upload finished. err: <nil> Aug 7 12:09:43.798285096 (2h29m43.387630083s) [upload=2616]: Starting upload Aug 7 12:09:43.798345140 (2h29m43.387690127s) [upload=2616]: Done starting upload Aug 7 12:09:43.798408411 (2h29m43.387753398s) [upload=2616]: Uploading segments... Aug 7 12:09:43.798439370 (2h29m43.387784357s) [upload=2616]: Got next segment. Inline: true Aug 7 12:09:43.798443217 (2h29m43.387788204s) [upload=2616]: Waiting for error group managing segments... Aug 7 12:09:43.798475981 (2h29m43.387820968s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:43.798479978 (2h29m43.387824965s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:43.798482313 (2h29m43.38782729s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:44.143582671 (2h29m43.732927658s) [upload=2616]: Done uploading segments... Aug 7 12:09:44.143598691 (2h29m43.732943668s) [upload=2616]: Upload finished. err: <nil> Aug 7 12:10:27.584576925 (2h30m27.173921912s) [upload=2629]: Starting upload Aug 7 12:10:27.584629195 (2h30m27.173974172s) [upload=2629]: Done starting upload Aug 7 12:10:27.584678159 (2h30m27.174023146s) [upload=2629]: Uploading segments... Aug 7 12:10:27.584707415 (2h30m27.174052402s) [upload=2629]: Got next segment. Inline: true Aug 7 12:10:27.584711312 (2h30m27.174056299s) [upload=2629]: Waiting for error group managing segments... Aug 7 12:10:27.584743203 (2h30m27.17408819s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:10:27.584747040 (2h30m27.174092027s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:10:27.584749435 (2h30m27.174094422s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:10:27.887810742 (2h30m27.477155729s) [upload=2629]: Done uploading segments... Aug 7 12:10:27.887828937 (2h30m27.477173914s) [upload=2629]: Upload finished. err: <nil> Aug 7 12:11:12.432073435 (2h31m12.021418422s) [upload=2642]: Starting upload Aug 7 12:11:12.432118301 (2h31m12.021463278s) [upload=2642]: Done starting upload Aug 7 12:11:12.432168257 (2h31m12.021513244s) [upload=2642]: Uploading segments... Aug 7 12:11:12.432197673 (2h31m12.02154266s) [upload=2642]: Got next segment. Inline: true Aug 7 12:11:12.432201360 (2h31m12.021546347s) [upload=2642]: Waiting for error group managing segments... Aug 7 12:11:12.432235225 (2h31m12.021580212s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:11:12.432238791 (2h31m12.021583778s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:11:12.432241005 (2h31m12.021585992s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:11:12.826402410 (2h31m12.415747397s) [upload=2642]: Done uploading segments... Aug 7 12:11:12.826420154 (2h31m12.415765141s) [upload=2642]: Upload finished. err: <nil> Aug 7 12:12:01.525617822 (2h32m1.114962819s) [upload=2656]: Starting upload Aug 7 12:12:01.525663079 (2h32m1.115008066s) [upload=2656]: Done starting upload Aug 7 12:12:01.525738163 (2h32m1.11508316s) [upload=2656]: Uploading segments... Aug 7 12:12:01.525854065 (2h32m1.115199052s) [upload=2656]: Got next segment. Inline: true Aug 7 12:12:01.525860697 (2h32m1.115205684s) [upload=2656]: Waiting for error group managing segments... Aug 7 12:12:01.525912907 (2h32m1.115257894s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:01.525919960 (2h32m1.115264957s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:01.525923287 (2h32m1.115268284s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:01.969933727 (2h32m1.559278714s) [upload=2656]: Done uploading segments... Aug 7 12:12:01.969966339 (2h32m1.559311326s) [upload=2656]: Upload finished. err: <nil> Aug 7 12:12:23.526243134 (2h32m23.115588181s) [upload=2664]: Starting upload Aug 7 12:12:23.526339669 (2h32m23.115684696s) [upload=2664]: Done starting upload Aug 7 12:12:23.526447936 (2h32m23.115792923s) [upload=2664]: Uploading segments... Aug 7 12:12:23.526522438 (2h32m23.115867445s) [upload=2664]: Got next segment. Inline: true Aug 7 12:12:23.526530534 (2h32m23.115875551s) [upload=2664]: Waiting for error group managing segments... Aug 7 12:12:23.526585439 (2h32m23.115930446s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:23.526592743 (2h32m23.11593775s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:23.526597251 (2h32m23.115942248s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:23.902417816 (2h32m23.491762803s) [upload=2664]: Done uploading segments... Aug 7 12:12:23.902436842 (2h32m23.491781829s) [upload=2664]: Upload finished. err: <nil> Aug 7 12:12:56.180563006 (2h32m55.769907993s) [upload=2668]: Starting upload Aug 7 12:12:56.180605848 (2h32m55.769950835s) [upload=2668]: Done starting upload Aug 7 12:12:56.180665191 (2h32m55.770010178s) [upload=2668]: Uploading segments... Aug 7 12:12:56.180703014 (2h32m55.770048001s) [upload=2668]: Got next segment. Inline: true Aug 7 12:12:56.180706641 (2h32m55.770051618s) [upload=2668]: Waiting for error group managing segments... Aug 7 12:12:56.180740676 (2h32m55.770085663s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:56.180744884 (2h32m55.770089871s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:56.180747298 (2h32m55.770092285s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:56.490488305 (2h32m56.079833292s) [upload=2668]: Done uploading segments... Aug 7 12:12:56.490511128 (2h32m56.079856115s) [upload=2668]: Upload finished. err: <nil> Aug 7 12:13:11.317339795 (2h33m10.906684772s) [upload=2675]: Starting upload Aug 7 12:13:11.317388448 (2h33m10.906733435s) [upload=2675]: Done starting upload Aug 7 12:13:11.317435919 (2h33m10.906780906s) [upload=2675]: Uploading segments... Aug 7 12:13:11.317477428 (2h33m10.906822415s) [upload=2675]: Got next segment. Inline: true Aug 7 12:13:11.317481295 (2h33m10.906826283s) [upload=2675]: Waiting for error group managing segments... Aug 7 12:13:11.317516183 (2h33m10.90686117s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:11.317520180 (2h33m10.906865168s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil>
Duration: 1.939185665s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
5.880765852s
FAIL (2025-08-07 12:17:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.880799266s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
5.769956319s
FAIL (2025-08-07 12:16:50 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.769980434s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....X...............................................X.......] duration:
1.854091335s
FAIL (2025-08-07 12:16:49 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after df11e3eae1ac8f408d918f55d929d54d4e5df3d8eaae7e4f8e4c7c8038d1c94c != before bd9ccba63d0a10a25cb9e58291141281665e96854a4357fa64c5bb1ae08d3d6f ======LOGS====== *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:06:53.059666435 (2h26m52.649011472s) [upload=2569]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:06:53.386093536 (2h26m52.975438523s) [upload=2569]: Done uploading segments... Aug 7 12:06:53.386195922 (2h26m52.975540909s) [upload=2569]: Upload finished. err: <nil> Aug 7 12:07:12.996984187 (2h27m12.586329175s) [upload=2577]: Starting upload Aug 7 12:07:12.997029885 (2h27m12.586374872s) [upload=2577]: Done starting upload Aug 7 12:07:12.997091613 (2h27m12.5864366s) [upload=2577]: Uploading segments... Aug 7 12:07:12.997174532 (2h27m12.586519509s) [upload=2577]: Got next segment. Inline: true Aug 7 12:07:12.997179731 (2h27m12.586524719s) [upload=2577]: Waiting for error group managing segments... Aug 7 12:07:12.997230749 (2h27m12.586575726s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:12.997236009 (2h27m12.586580996s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:12.997238444 (2h27m12.586583431s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:13.277508374 (2h27m12.866853361s) [upload=2577]: Done uploading segments... Aug 7 12:07:13.277529685 (2h27m12.866874672s) [upload=2577]: Upload finished. err: <nil> Aug 7 12:07:59.647444303 (2h27m59.23678929s) [upload=2586]: Starting upload Aug 7 12:07:59.647485382 (2h27m59.236830369s) [upload=2586]: Done starting upload Aug 7 12:07:59.647544936 (2h27m59.236889923s) [upload=2586]: Uploading segments... Aug 7 12:07:59.647582257 (2h27m59.236927234s) [upload=2586]: Got next segment. Inline: true Aug 7 12:07:59.647585894 (2h27m59.236930881s) [upload=2586]: Waiting for error group managing segments... Aug 7 12:07:59.647649967 (2h27m59.236994954s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:59.647654155 (2h27m59.236999142s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:59.647656389 (2h27m59.237001376s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:59.961215120 (2h27m59.550560107s) [upload=2586]: Done uploading segments... Aug 7 12:07:59.961233604 (2h27m59.550578581s) [upload=2586]: Upload finished. err: <nil> Aug 7 12:09:04.474008716 (2h29m4.063353693s) [upload=2606]: Starting upload Aug 7 12:09:04.474054282 (2h29m4.063399269s) [upload=2606]: Done starting upload Aug 7 12:09:04.474117544 (2h29m4.063462531s) [upload=2606]: Uploading segments... Aug 7 12:09:04.474146590 (2h29m4.063491567s) [upload=2606]: Got next segment. Inline: true Aug 7 12:09:04.474149866 (2h29m4.063494853s) [upload=2606]: Waiting for error group managing segments... Aug 7 12:09:04.474185023 (2h29m4.06353001s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:04.474190423 (2h29m4.0635354s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:04.474193168 (2h29m4.063538145s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:04.800005844 (2h29m4.389350831s) [upload=2606]: Done uploading segments... Aug 7 12:09:04.800024600 (2h29m4.389369587s) [upload=2606]: Upload finished. err: <nil> Aug 7 12:09:43.798285096 (2h29m43.387630083s) [upload=2616]: Starting upload Aug 7 12:09:43.798345140 (2h29m43.387690127s) [upload=2616]: Done starting upload Aug 7 12:09:43.798408411 (2h29m43.387753398s) [upload=2616]: Uploading segments... Aug 7 12:09:43.798439370 (2h29m43.387784357s) [upload=2616]: Got next segment. Inline: true Aug 7 12:09:43.798443217 (2h29m43.387788204s) [upload=2616]: Waiting for error group managing segments... Aug 7 12:09:43.798475981 (2h29m43.387820968s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:43.798479978 (2h29m43.387824965s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:43.798482313 (2h29m43.38782729s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:44.143582671 (2h29m43.732927658s) [upload=2616]: Done uploading segments... Aug 7 12:09:44.143598691 (2h29m43.732943668s) [upload=2616]: Upload finished. err: <nil> Aug 7 12:10:27.584576925 (2h30m27.173921912s) [upload=2629]: Starting upload Aug 7 12:10:27.584629195 (2h30m27.173974172s) [upload=2629]: Done starting upload Aug 7 12:10:27.584678159 (2h30m27.174023146s) [upload=2629]: Uploading segments... Aug 7 12:10:27.584707415 (2h30m27.174052402s) [upload=2629]: Got next segment. Inline: true Aug 7 12:10:27.584711312 (2h30m27.174056299s) [upload=2629]: Waiting for error group managing segments... Aug 7 12:10:27.584743203 (2h30m27.17408819s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:10:27.584747040 (2h30m27.174092027s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:10:27.584749435 (2h30m27.174094422s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:10:27.887810742 (2h30m27.477155729s) [upload=2629]: Done uploading segments... Aug 7 12:10:27.887828937 (2h30m27.477173914s) [upload=2629]: Upload finished. err: <nil> Aug 7 12:11:12.432073435 (2h31m12.021418422s) [upload=2642]: Starting upload Aug 7 12:11:12.432118301 (2h31m12.021463278s) [upload=2642]: Done starting upload Aug 7 12:11:12.432168257 (2h31m12.021513244s) [upload=2642]: Uploading segments... Aug 7 12:11:12.432197673 (2h31m12.02154266s) [upload=2642]: Got next segment. Inline: true Aug 7 12:11:12.432201360 (2h31m12.021546347s) [upload=2642]: Waiting for error group managing segments... Aug 7 12:11:12.432235225 (2h31m12.021580212s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:11:12.432238791 (2h31m12.021583778s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:11:12.432241005 (2h31m12.021585992s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:11:12.826402410 (2h31m12.415747397s) [upload=2642]: Done uploading segments... Aug 7 12:11:12.826420154 (2h31m12.415765141s) [upload=2642]: Upload finished. err: <nil> Aug 7 12:12:01.525617822 (2h32m1.114962819s) [upload=2656]: Starting upload Aug 7 12:12:01.525663079 (2h32m1.115008066s) [upload=2656]: Done starting upload Aug 7 12:12:01.525738163 (2h32m1.11508316s) [upload=2656]: Uploading segments... Aug 7 12:12:01.525854065 (2h32m1.115199052s) [upload=2656]: Got next segment. Inline: true Aug 7 12:12:01.525860697 (2h32m1.115205684s) [upload=2656]: Waiting for error group managing segments... Aug 7 12:12:01.525912907 (2h32m1.115257894s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:01.525919960 (2h32m1.115264957s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:01.525923287 (2h32m1.115268284s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:01.969933727 (2h32m1.559278714s) [upload=2656]: Done uploading segments... Aug 7 12:12:01.969966339 (2h32m1.559311326s) [upload=2656]: Upload finished. err: <nil> Aug 7 12:12:23.526243134 (2h32m23.115588181s) [upload=2664]: Starting upload Aug 7 12:12:23.526339669 (2h32m23.115684696s) [upload=2664]: Done starting upload Aug 7 12:12:23.526447936 (2h32m23.115792923s) [upload=2664]: Uploading segments... Aug 7 12:12:23.526522438 (2h32m23.115867445s) [upload=2664]: Got next segment. Inline: true Aug 7 12:12:23.526530534 (2h32m23.115875551s) [upload=2664]: Waiting for error group managing segments... Aug 7 12:12:23.526585439 (2h32m23.115930446s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:23.526592743 (2h32m23.11593775s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:23.526597251 (2h32m23.115942248s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:23.902417816 (2h32m23.491762803s) [upload=2664]: Done uploading segments... Aug 7 12:12:23.902436842 (2h32m23.491781829s) [upload=2664]: Upload finished. err: <nil> Aug 7 12:12:56.180563006 (2h32m55.769907993s) [upload=2668]: Starting upload Aug 7 12:12:56.180605848 (2h32m55.769950835s) [upload=2668]: Done starting upload Aug 7 12:12:56.180665191 (2h32m55.770010178s) [upload=2668]: Uploading segments... Aug 7 12:12:56.180703014 (2h32m55.770048001s) [upload=2668]: Got next segment. Inline: true Aug 7 12:12:56.180706641 (2h32m55.770051618s) [upload=2668]: Waiting for error group managing segments... Aug 7 12:12:56.180740676 (2h32m55.770085663s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:56.180744884 (2h32m55.770089871s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:56.180747298 (2h32m55.770092285s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:56.490488305 (2h32m56.079833292s) [upload=2668]: Done uploading segments... Aug 7 12:12:56.490511128 (2h32m56.079856115s) [upload=2668]: Upload finished. err: <nil> Aug 7 12:13:11.317339795 (2h33m10.906684772s) [upload=2675]: Starting upload Aug 7 12:13:11.317388448 (2h33m10.906733435s) [upload=2675]: Done starting upload Aug 7 12:13:11.317435919 (2h33m10.906780906s) [upload=2675]: Uploading segments... Aug 7 12:13:11.317477428 (2h33m10.906822415s) [upload=2675]: Got next segment. Inline: true Aug 7 12:13:11.317481295 (2h33m10.906826283s) [upload=2675]: Waiting for error group managing segments... Aug 7 12:13:11.317516183 (2h33m10.90686117s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:11.317520180 (2h33m10.906865168s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil>
Duration: 1.854116544s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....X...............................................X......] duration:
1.986348628s
FAIL (2025-08-07 12:16:29 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after df11e3eae1ac8f408d918f55d929d54d4e5df3d8eaae7e4f8e4c7c8038d1c94c != before bd9ccba63d0a10a25cb9e58291141281665e96854a4357fa64c5bb1ae08d3d6f ======LOGS====== *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:06:53.059666435 (2h26m52.649011472s) [upload=2569]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:06:53.386093536 (2h26m52.975438523s) [upload=2569]: Done uploading segments... Aug 7 12:06:53.386195922 (2h26m52.975540909s) [upload=2569]: Upload finished. err: <nil> Aug 7 12:07:12.996984187 (2h27m12.586329175s) [upload=2577]: Starting upload Aug 7 12:07:12.997029885 (2h27m12.586374872s) [upload=2577]: Done starting upload Aug 7 12:07:12.997091613 (2h27m12.5864366s) [upload=2577]: Uploading segments... Aug 7 12:07:12.997174532 (2h27m12.586519509s) [upload=2577]: Got next segment. Inline: true Aug 7 12:07:12.997179731 (2h27m12.586524719s) [upload=2577]: Waiting for error group managing segments... Aug 7 12:07:12.997230749 (2h27m12.586575726s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:12.997236009 (2h27m12.586580996s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:12.997238444 (2h27m12.586583431s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:13.277508374 (2h27m12.866853361s) [upload=2577]: Done uploading segments... Aug 7 12:07:13.277529685 (2h27m12.866874672s) [upload=2577]: Upload finished. err: <nil> Aug 7 12:07:59.647444303 (2h27m59.23678929s) [upload=2586]: Starting upload Aug 7 12:07:59.647485382 (2h27m59.236830369s) [upload=2586]: Done starting upload Aug 7 12:07:59.647544936 (2h27m59.236889923s) [upload=2586]: Uploading segments... Aug 7 12:07:59.647582257 (2h27m59.236927234s) [upload=2586]: Got next segment. Inline: true Aug 7 12:07:59.647585894 (2h27m59.236930881s) [upload=2586]: Waiting for error group managing segments... Aug 7 12:07:59.647649967 (2h27m59.236994954s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:59.647654155 (2h27m59.236999142s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:59.647656389 (2h27m59.237001376s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:59.961215120 (2h27m59.550560107s) [upload=2586]: Done uploading segments... Aug 7 12:07:59.961233604 (2h27m59.550578581s) [upload=2586]: Upload finished. err: <nil> Aug 7 12:09:04.474008716 (2h29m4.063353693s) [upload=2606]: Starting upload Aug 7 12:09:04.474054282 (2h29m4.063399269s) [upload=2606]: Done starting upload Aug 7 12:09:04.474117544 (2h29m4.063462531s) [upload=2606]: Uploading segments... Aug 7 12:09:04.474146590 (2h29m4.063491567s) [upload=2606]: Got next segment. Inline: true Aug 7 12:09:04.474149866 (2h29m4.063494853s) [upload=2606]: Waiting for error group managing segments... Aug 7 12:09:04.474185023 (2h29m4.06353001s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:04.474190423 (2h29m4.0635354s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:04.474193168 (2h29m4.063538145s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:04.800005844 (2h29m4.389350831s) [upload=2606]: Done uploading segments... Aug 7 12:09:04.800024600 (2h29m4.389369587s) [upload=2606]: Upload finished. err: <nil> Aug 7 12:09:43.798285096 (2h29m43.387630083s) [upload=2616]: Starting upload Aug 7 12:09:43.798345140 (2h29m43.387690127s) [upload=2616]: Done starting upload Aug 7 12:09:43.798408411 (2h29m43.387753398s) [upload=2616]: Uploading segments... Aug 7 12:09:43.798439370 (2h29m43.387784357s) [upload=2616]: Got next segment. Inline: true Aug 7 12:09:43.798443217 (2h29m43.387788204s) [upload=2616]: Waiting for error group managing segments... Aug 7 12:09:43.798475981 (2h29m43.387820968s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:43.798479978 (2h29m43.387824965s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:43.798482313 (2h29m43.38782729s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:44.143582671 (2h29m43.732927658s) [upload=2616]: Done uploading segments... Aug 7 12:09:44.143598691 (2h29m43.732943668s) [upload=2616]: Upload finished. err: <nil> Aug 7 12:10:27.584576925 (2h30m27.173921912s) [upload=2629]: Starting upload Aug 7 12:10:27.584629195 (2h30m27.173974172s) [upload=2629]: Done starting upload Aug 7 12:10:27.584678159 (2h30m27.174023146s) [upload=2629]: Uploading segments... Aug 7 12:10:27.584707415 (2h30m27.174052402s) [upload=2629]: Got next segment. Inline: true Aug 7 12:10:27.584711312 (2h30m27.174056299s) [upload=2629]: Waiting for error group managing segments... Aug 7 12:10:27.584743203 (2h30m27.17408819s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:10:27.584747040 (2h30m27.174092027s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:10:27.584749435 (2h30m27.174094422s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:10:27.887810742 (2h30m27.477155729s) [upload=2629]: Done uploading segments... Aug 7 12:10:27.887828937 (2h30m27.477173914s) [upload=2629]: Upload finished. err: <nil> Aug 7 12:11:12.432073435 (2h31m12.021418422s) [upload=2642]: Starting upload Aug 7 12:11:12.432118301 (2h31m12.021463278s) [upload=2642]: Done starting upload Aug 7 12:11:12.432168257 (2h31m12.021513244s) [upload=2642]: Uploading segments... Aug 7 12:11:12.432197673 (2h31m12.02154266s) [upload=2642]: Got next segment. Inline: true Aug 7 12:11:12.432201360 (2h31m12.021546347s) [upload=2642]: Waiting for error group managing segments... Aug 7 12:11:12.432235225 (2h31m12.021580212s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:11:12.432238791 (2h31m12.021583778s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:11:12.432241005 (2h31m12.021585992s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:11:12.826402410 (2h31m12.415747397s) [upload=2642]: Done uploading segments... Aug 7 12:11:12.826420154 (2h31m12.415765141s) [upload=2642]: Upload finished. err: <nil> Aug 7 12:12:01.525617822 (2h32m1.114962819s) [upload=2656]: Starting upload Aug 7 12:12:01.525663079 (2h32m1.115008066s) [upload=2656]: Done starting upload Aug 7 12:12:01.525738163 (2h32m1.11508316s) [upload=2656]: Uploading segments... Aug 7 12:12:01.525854065 (2h32m1.115199052s) [upload=2656]: Got next segment. Inline: true Aug 7 12:12:01.525860697 (2h32m1.115205684s) [upload=2656]: Waiting for error group managing segments... Aug 7 12:12:01.525912907 (2h32m1.115257894s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:01.525919960 (2h32m1.115264957s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:01.525923287 (2h32m1.115268284s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:01.969933727 (2h32m1.559278714s) [upload=2656]: Done uploading segments... Aug 7 12:12:01.969966339 (2h32m1.559311326s) [upload=2656]: Upload finished. err: <nil> Aug 7 12:12:23.526243134 (2h32m23.115588181s) [upload=2664]: Starting upload Aug 7 12:12:23.526339669 (2h32m23.115684696s) [upload=2664]: Done starting upload Aug 7 12:12:23.526447936 (2h32m23.115792923s) [upload=2664]: Uploading segments... Aug 7 12:12:23.526522438 (2h32m23.115867445s) [upload=2664]: Got next segment. Inline: true Aug 7 12:12:23.526530534 (2h32m23.115875551s) [upload=2664]: Waiting for error group managing segments... Aug 7 12:12:23.526585439 (2h32m23.115930446s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:23.526592743 (2h32m23.11593775s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:23.526597251 (2h32m23.115942248s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:23.902417816 (2h32m23.491762803s) [upload=2664]: Done uploading segments... Aug 7 12:12:23.902436842 (2h32m23.491781829s) [upload=2664]: Upload finished. err: <nil> Aug 7 12:12:56.180563006 (2h32m55.769907993s) [upload=2668]: Starting upload Aug 7 12:12:56.180605848 (2h32m55.769950835s) [upload=2668]: Done starting upload Aug 7 12:12:56.180665191 (2h32m55.770010178s) [upload=2668]: Uploading segments... Aug 7 12:12:56.180703014 (2h32m55.770048001s) [upload=2668]: Got next segment. Inline: true Aug 7 12:12:56.180706641 (2h32m55.770051618s) [upload=2668]: Waiting for error group managing segments... Aug 7 12:12:56.180740676 (2h32m55.770085663s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:56.180744884 (2h32m55.770089871s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:56.180747298 (2h32m55.770092285s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:56.490488305 (2h32m56.079833292s) [upload=2668]: Done uploading segments... Aug 7 12:12:56.490511128 (2h32m56.079856115s) [upload=2668]: Upload finished. err: <nil> Aug 7 12:13:11.317339795 (2h33m10.906684772s) [upload=2675]: Starting upload Aug 7 12:13:11.317388448 (2h33m10.906733435s) [upload=2675]: Done starting upload Aug 7 12:13:11.317435919 (2h33m10.906780906s) [upload=2675]: Uploading segments... Aug 7 12:13:11.317477428 (2h33m10.906822415s) [upload=2675]: Got next segment. Inline: true Aug 7 12:13:11.317481295 (2h33m10.906826283s) [upload=2675]: Waiting for error group managing segments... Aug 7 12:13:11.317516183 (2h33m10.90686117s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:11.317520180 (2h33m10.906865168s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil>
Duration: 1.986376963s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.846688809s
FAIL (2025-08-07 12:16:16 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.84671584s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......X...............................................X.....] duration:
1.856482251s
FAIL (2025-08-07 12:16:13 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after df11e3eae1ac8f408d918f55d929d54d4e5df3d8eaae7e4f8e4c7c8038d1c94c != before bd9ccba63d0a10a25cb9e58291141281665e96854a4357fa64c5bb1ae08d3d6f ======LOGS====== *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:06:53.059666435 (2h26m52.649011472s) [upload=2569]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:06:53.386093536 (2h26m52.975438523s) [upload=2569]: Done uploading segments... Aug 7 12:06:53.386195922 (2h26m52.975540909s) [upload=2569]: Upload finished. err: <nil> Aug 7 12:07:12.996984187 (2h27m12.586329175s) [upload=2577]: Starting upload Aug 7 12:07:12.997029885 (2h27m12.586374872s) [upload=2577]: Done starting upload Aug 7 12:07:12.997091613 (2h27m12.5864366s) [upload=2577]: Uploading segments... Aug 7 12:07:12.997174532 (2h27m12.586519509s) [upload=2577]: Got next segment. Inline: true Aug 7 12:07:12.997179731 (2h27m12.586524719s) [upload=2577]: Waiting for error group managing segments... Aug 7 12:07:12.997230749 (2h27m12.586575726s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:12.997236009 (2h27m12.586580996s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:12.997238444 (2h27m12.586583431s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:13.277508374 (2h27m12.866853361s) [upload=2577]: Done uploading segments... Aug 7 12:07:13.277529685 (2h27m12.866874672s) [upload=2577]: Upload finished. err: <nil> Aug 7 12:07:59.647444303 (2h27m59.23678929s) [upload=2586]: Starting upload Aug 7 12:07:59.647485382 (2h27m59.236830369s) [upload=2586]: Done starting upload Aug 7 12:07:59.647544936 (2h27m59.236889923s) [upload=2586]: Uploading segments... Aug 7 12:07:59.647582257 (2h27m59.236927234s) [upload=2586]: Got next segment. Inline: true Aug 7 12:07:59.647585894 (2h27m59.236930881s) [upload=2586]: Waiting for error group managing segments... Aug 7 12:07:59.647649967 (2h27m59.236994954s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:59.647654155 (2h27m59.236999142s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:59.647656389 (2h27m59.237001376s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:59.961215120 (2h27m59.550560107s) [upload=2586]: Done uploading segments... Aug 7 12:07:59.961233604 (2h27m59.550578581s) [upload=2586]: Upload finished. err: <nil> Aug 7 12:09:04.474008716 (2h29m4.063353693s) [upload=2606]: Starting upload Aug 7 12:09:04.474054282 (2h29m4.063399269s) [upload=2606]: Done starting upload Aug 7 12:09:04.474117544 (2h29m4.063462531s) [upload=2606]: Uploading segments... Aug 7 12:09:04.474146590 (2h29m4.063491567s) [upload=2606]: Got next segment. Inline: true Aug 7 12:09:04.474149866 (2h29m4.063494853s) [upload=2606]: Waiting for error group managing segments... Aug 7 12:09:04.474185023 (2h29m4.06353001s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:04.474190423 (2h29m4.0635354s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:04.474193168 (2h29m4.063538145s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:04.800005844 (2h29m4.389350831s) [upload=2606]: Done uploading segments... Aug 7 12:09:04.800024600 (2h29m4.389369587s) [upload=2606]: Upload finished. err: <nil> Aug 7 12:09:43.798285096 (2h29m43.387630083s) [upload=2616]: Starting upload Aug 7 12:09:43.798345140 (2h29m43.387690127s) [upload=2616]: Done starting upload Aug 7 12:09:43.798408411 (2h29m43.387753398s) [upload=2616]: Uploading segments... Aug 7 12:09:43.798439370 (2h29m43.387784357s) [upload=2616]: Got next segment. Inline: true Aug 7 12:09:43.798443217 (2h29m43.387788204s) [upload=2616]: Waiting for error group managing segments... Aug 7 12:09:43.798475981 (2h29m43.387820968s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:43.798479978 (2h29m43.387824965s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:43.798482313 (2h29m43.38782729s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:44.143582671 (2h29m43.732927658s) [upload=2616]: Done uploading segments... Aug 7 12:09:44.143598691 (2h29m43.732943668s) [upload=2616]: Upload finished. err: <nil> Aug 7 12:10:27.584576925 (2h30m27.173921912s) [upload=2629]: Starting upload Aug 7 12:10:27.584629195 (2h30m27.173974172s) [upload=2629]: Done starting upload Aug 7 12:10:27.584678159 (2h30m27.174023146s) [upload=2629]: Uploading segments... Aug 7 12:10:27.584707415 (2h30m27.174052402s) [upload=2629]: Got next segment. Inline: true Aug 7 12:10:27.584711312 (2h30m27.174056299s) [upload=2629]: Waiting for error group managing segments... Aug 7 12:10:27.584743203 (2h30m27.17408819s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:10:27.584747040 (2h30m27.174092027s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:10:27.584749435 (2h30m27.174094422s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:10:27.887810742 (2h30m27.477155729s) [upload=2629]: Done uploading segments... Aug 7 12:10:27.887828937 (2h30m27.477173914s) [upload=2629]: Upload finished. err: <nil> Aug 7 12:11:12.432073435 (2h31m12.021418422s) [upload=2642]: Starting upload Aug 7 12:11:12.432118301 (2h31m12.021463278s) [upload=2642]: Done starting upload Aug 7 12:11:12.432168257 (2h31m12.021513244s) [upload=2642]: Uploading segments... Aug 7 12:11:12.432197673 (2h31m12.02154266s) [upload=2642]: Got next segment. Inline: true Aug 7 12:11:12.432201360 (2h31m12.021546347s) [upload=2642]: Waiting for error group managing segments... Aug 7 12:11:12.432235225 (2h31m12.021580212s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:11:12.432238791 (2h31m12.021583778s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:11:12.432241005 (2h31m12.021585992s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:11:12.826402410 (2h31m12.415747397s) [upload=2642]: Done uploading segments... Aug 7 12:11:12.826420154 (2h31m12.415765141s) [upload=2642]: Upload finished. err: <nil> Aug 7 12:12:01.525617822 (2h32m1.114962819s) [upload=2656]: Starting upload Aug 7 12:12:01.525663079 (2h32m1.115008066s) [upload=2656]: Done starting upload Aug 7 12:12:01.525738163 (2h32m1.11508316s) [upload=2656]: Uploading segments... Aug 7 12:12:01.525854065 (2h32m1.115199052s) [upload=2656]: Got next segment. Inline: true Aug 7 12:12:01.525860697 (2h32m1.115205684s) [upload=2656]: Waiting for error group managing segments... Aug 7 12:12:01.525912907 (2h32m1.115257894s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:01.525919960 (2h32m1.115264957s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:01.525923287 (2h32m1.115268284s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:01.969933727 (2h32m1.559278714s) [upload=2656]: Done uploading segments... Aug 7 12:12:01.969966339 (2h32m1.559311326s) [upload=2656]: Upload finished. err: <nil> Aug 7 12:12:23.526243134 (2h32m23.115588181s) [upload=2664]: Starting upload Aug 7 12:12:23.526339669 (2h32m23.115684696s) [upload=2664]: Done starting upload Aug 7 12:12:23.526447936 (2h32m23.115792923s) [upload=2664]: Uploading segments... Aug 7 12:12:23.526522438 (2h32m23.115867445s) [upload=2664]: Got next segment. Inline: true Aug 7 12:12:23.526530534 (2h32m23.115875551s) [upload=2664]: Waiting for error group managing segments... Aug 7 12:12:23.526585439 (2h32m23.115930446s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:23.526592743 (2h32m23.11593775s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:23.526597251 (2h32m23.115942248s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:23.902417816 (2h32m23.491762803s) [upload=2664]: Done uploading segments... Aug 7 12:12:23.902436842 (2h32m23.491781829s) [upload=2664]: Upload finished. err: <nil> Aug 7 12:12:56.180563006 (2h32m55.769907993s) [upload=2668]: Starting upload Aug 7 12:12:56.180605848 (2h32m55.769950835s) [upload=2668]: Done starting upload Aug 7 12:12:56.180665191 (2h32m55.770010178s) [upload=2668]: Uploading segments... Aug 7 12:12:56.180703014 (2h32m55.770048001s) [upload=2668]: Got next segment. Inline: true Aug 7 12:12:56.180706641 (2h32m55.770051618s) [upload=2668]: Waiting for error group managing segments... Aug 7 12:12:56.180740676 (2h32m55.770085663s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:56.180744884 (2h32m55.770089871s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:56.180747298 (2h32m55.770092285s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:56.490488305 (2h32m56.079833292s) [upload=2668]: Done uploading segments... Aug 7 12:12:56.490511128 (2h32m56.079856115s) [upload=2668]: Upload finished. err: <nil> Aug 7 12:13:11.317339795 (2h33m10.906684772s) [upload=2675]: Starting upload Aug 7 12:13:11.317388448 (2h33m10.906733435s) [upload=2675]: Done starting upload Aug 7 12:13:11.317435919 (2h33m10.906780906s) [upload=2675]: Uploading segments... Aug 7 12:13:11.317477428 (2h33m10.906822415s) [upload=2675]: Got next segment. Inline: true Aug 7 12:13:11.317481295 (2h33m10.906826283s) [upload=2675]: Waiting for error group managing segments... Aug 7 12:13:11.317516183 (2h33m10.90686117s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:11.317520180 (2h33m10.906865168s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil>
Duration: 1.856510945s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......X...............................................X....] duration:
1.935670298s
FAIL (2025-08-07 12:15:42 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after df11e3eae1ac8f408d918f55d929d54d4e5df3d8eaae7e4f8e4c7c8038d1c94c != before bd9ccba63d0a10a25cb9e58291141281665e96854a4357fa64c5bb1ae08d3d6f ======LOGS====== *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:06:53.059666435 (2h26m52.649011472s) [upload=2569]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:06:53.386093536 (2h26m52.975438523s) [upload=2569]: Done uploading segments... Aug 7 12:06:53.386195922 (2h26m52.975540909s) [upload=2569]: Upload finished. err: <nil> Aug 7 12:07:12.996984187 (2h27m12.586329175s) [upload=2577]: Starting upload Aug 7 12:07:12.997029885 (2h27m12.586374872s) [upload=2577]: Done starting upload Aug 7 12:07:12.997091613 (2h27m12.5864366s) [upload=2577]: Uploading segments... Aug 7 12:07:12.997174532 (2h27m12.586519509s) [upload=2577]: Got next segment. Inline: true Aug 7 12:07:12.997179731 (2h27m12.586524719s) [upload=2577]: Waiting for error group managing segments... Aug 7 12:07:12.997230749 (2h27m12.586575726s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:12.997236009 (2h27m12.586580996s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:12.997238444 (2h27m12.586583431s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:13.277508374 (2h27m12.866853361s) [upload=2577]: Done uploading segments... Aug 7 12:07:13.277529685 (2h27m12.866874672s) [upload=2577]: Upload finished. err: <nil> Aug 7 12:07:59.647444303 (2h27m59.23678929s) [upload=2586]: Starting upload Aug 7 12:07:59.647485382 (2h27m59.236830369s) [upload=2586]: Done starting upload Aug 7 12:07:59.647544936 (2h27m59.236889923s) [upload=2586]: Uploading segments... Aug 7 12:07:59.647582257 (2h27m59.236927234s) [upload=2586]: Got next segment. Inline: true Aug 7 12:07:59.647585894 (2h27m59.236930881s) [upload=2586]: Waiting for error group managing segments... Aug 7 12:07:59.647649967 (2h27m59.236994954s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:59.647654155 (2h27m59.236999142s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:59.647656389 (2h27m59.237001376s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:59.961215120 (2h27m59.550560107s) [upload=2586]: Done uploading segments... Aug 7 12:07:59.961233604 (2h27m59.550578581s) [upload=2586]: Upload finished. err: <nil> Aug 7 12:09:04.474008716 (2h29m4.063353693s) [upload=2606]: Starting upload Aug 7 12:09:04.474054282 (2h29m4.063399269s) [upload=2606]: Done starting upload Aug 7 12:09:04.474117544 (2h29m4.063462531s) [upload=2606]: Uploading segments... Aug 7 12:09:04.474146590 (2h29m4.063491567s) [upload=2606]: Got next segment. Inline: true Aug 7 12:09:04.474149866 (2h29m4.063494853s) [upload=2606]: Waiting for error group managing segments... Aug 7 12:09:04.474185023 (2h29m4.06353001s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:04.474190423 (2h29m4.0635354s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:04.474193168 (2h29m4.063538145s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:04.800005844 (2h29m4.389350831s) [upload=2606]: Done uploading segments... Aug 7 12:09:04.800024600 (2h29m4.389369587s) [upload=2606]: Upload finished. err: <nil> Aug 7 12:09:43.798285096 (2h29m43.387630083s) [upload=2616]: Starting upload Aug 7 12:09:43.798345140 (2h29m43.387690127s) [upload=2616]: Done starting upload Aug 7 12:09:43.798408411 (2h29m43.387753398s) [upload=2616]: Uploading segments... Aug 7 12:09:43.798439370 (2h29m43.387784357s) [upload=2616]: Got next segment. Inline: true Aug 7 12:09:43.798443217 (2h29m43.387788204s) [upload=2616]: Waiting for error group managing segments... Aug 7 12:09:43.798475981 (2h29m43.387820968s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:43.798479978 (2h29m43.387824965s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:43.798482313 (2h29m43.38782729s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:44.143582671 (2h29m43.732927658s) [upload=2616]: Done uploading segments... Aug 7 12:09:44.143598691 (2h29m43.732943668s) [upload=2616]: Upload finished. err: <nil> Aug 7 12:10:27.584576925 (2h30m27.173921912s) [upload=2629]: Starting upload Aug 7 12:10:27.584629195 (2h30m27.173974172s) [upload=2629]: Done starting upload Aug 7 12:10:27.584678159 (2h30m27.174023146s) [upload=2629]: Uploading segments... Aug 7 12:10:27.584707415 (2h30m27.174052402s) [upload=2629]: Got next segment. Inline: true Aug 7 12:10:27.584711312 (2h30m27.174056299s) [upload=2629]: Waiting for error group managing segments... Aug 7 12:10:27.584743203 (2h30m27.17408819s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:10:27.584747040 (2h30m27.174092027s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:10:27.584749435 (2h30m27.174094422s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:10:27.887810742 (2h30m27.477155729s) [upload=2629]: Done uploading segments... Aug 7 12:10:27.887828937 (2h30m27.477173914s) [upload=2629]: Upload finished. err: <nil> Aug 7 12:11:12.432073435 (2h31m12.021418422s) [upload=2642]: Starting upload Aug 7 12:11:12.432118301 (2h31m12.021463278s) [upload=2642]: Done starting upload Aug 7 12:11:12.432168257 (2h31m12.021513244s) [upload=2642]: Uploading segments... Aug 7 12:11:12.432197673 (2h31m12.02154266s) [upload=2642]: Got next segment. Inline: true Aug 7 12:11:12.432201360 (2h31m12.021546347s) [upload=2642]: Waiting for error group managing segments... Aug 7 12:11:12.432235225 (2h31m12.021580212s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:11:12.432238791 (2h31m12.021583778s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:11:12.432241005 (2h31m12.021585992s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:11:12.826402410 (2h31m12.415747397s) [upload=2642]: Done uploading segments... Aug 7 12:11:12.826420154 (2h31m12.415765141s) [upload=2642]: Upload finished. err: <nil> Aug 7 12:12:01.525617822 (2h32m1.114962819s) [upload=2656]: Starting upload Aug 7 12:12:01.525663079 (2h32m1.115008066s) [upload=2656]: Done starting upload Aug 7 12:12:01.525738163 (2h32m1.11508316s) [upload=2656]: Uploading segments... Aug 7 12:12:01.525854065 (2h32m1.115199052s) [upload=2656]: Got next segment. Inline: true Aug 7 12:12:01.525860697 (2h32m1.115205684s) [upload=2656]: Waiting for error group managing segments... Aug 7 12:12:01.525912907 (2h32m1.115257894s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:01.525919960 (2h32m1.115264957s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:01.525923287 (2h32m1.115268284s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:01.969933727 (2h32m1.559278714s) [upload=2656]: Done uploading segments... Aug 7 12:12:01.969966339 (2h32m1.559311326s) [upload=2656]: Upload finished. err: <nil> Aug 7 12:12:23.526243134 (2h32m23.115588181s) [upload=2664]: Starting upload Aug 7 12:12:23.526339669 (2h32m23.115684696s) [upload=2664]: Done starting upload Aug 7 12:12:23.526447936 (2h32m23.115792923s) [upload=2664]: Uploading segments... Aug 7 12:12:23.526522438 (2h32m23.115867445s) [upload=2664]: Got next segment. Inline: true Aug 7 12:12:23.526530534 (2h32m23.115875551s) [upload=2664]: Waiting for error group managing segments... Aug 7 12:12:23.526585439 (2h32m23.115930446s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:23.526592743 (2h32m23.11593775s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:23.526597251 (2h32m23.115942248s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:23.902417816 (2h32m23.491762803s) [upload=2664]: Done uploading segments... Aug 7 12:12:23.902436842 (2h32m23.491781829s) [upload=2664]: Upload finished. err: <nil> Aug 7 12:12:56.180563006 (2h32m55.769907993s) [upload=2668]: Starting upload Aug 7 12:12:56.180605848 (2h32m55.769950835s) [upload=2668]: Done starting upload Aug 7 12:12:56.180665191 (2h32m55.770010178s) [upload=2668]: Uploading segments... Aug 7 12:12:56.180703014 (2h32m55.770048001s) [upload=2668]: Got next segment. Inline: true Aug 7 12:12:56.180706641 (2h32m55.770051618s) [upload=2668]: Waiting for error group managing segments... Aug 7 12:12:56.180740676 (2h32m55.770085663s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:56.180744884 (2h32m55.770089871s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:56.180747298 (2h32m55.770092285s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:56.490488305 (2h32m56.079833292s) [upload=2668]: Done uploading segments... Aug 7 12:12:56.490511128 (2h32m56.079856115s) [upload=2668]: Upload finished. err: <nil> Aug 7 12:13:11.317339795 (2h33m10.906684772s) [upload=2675]: Starting upload Aug 7 12:13:11.317388448 (2h33m10.906733435s) [upload=2675]: Done starting upload Aug 7 12:13:11.317435919 (2h33m10.906780906s) [upload=2675]: Uploading segments... Aug 7 12:13:11.317477428 (2h33m10.906822415s) [upload=2675]: Got next segment. Inline: true Aug 7 12:13:11.317481295 (2h33m10.906826283s) [upload=2675]: Waiting for error group managing segments... Aug 7 12:13:11.317516183 (2h33m10.90686117s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:11.317520180 (2h33m10.906865168s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil>
Duration: 1.935701567s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
5.81044752s
FAIL (2025-08-07 12:15:25 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.810470433s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........X...............................................X...] duration:
1.966477009s
FAIL (2025-08-07 12:14:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after df11e3eae1ac8f408d918f55d929d54d4e5df3d8eaae7e4f8e4c7c8038d1c94c != before bd9ccba63d0a10a25cb9e58291141281665e96854a4357fa64c5bb1ae08d3d6f ======LOGS====== *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:06:53.059666435 (2h26m52.649011472s) [upload=2569]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:06:53.386093536 (2h26m52.975438523s) [upload=2569]: Done uploading segments... Aug 7 12:06:53.386195922 (2h26m52.975540909s) [upload=2569]: Upload finished. err: <nil> Aug 7 12:07:12.996984187 (2h27m12.586329175s) [upload=2577]: Starting upload Aug 7 12:07:12.997029885 (2h27m12.586374872s) [upload=2577]: Done starting upload Aug 7 12:07:12.997091613 (2h27m12.5864366s) [upload=2577]: Uploading segments... Aug 7 12:07:12.997174532 (2h27m12.586519509s) [upload=2577]: Got next segment. Inline: true Aug 7 12:07:12.997179731 (2h27m12.586524719s) [upload=2577]: Waiting for error group managing segments... Aug 7 12:07:12.997230749 (2h27m12.586575726s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:12.997236009 (2h27m12.586580996s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:12.997238444 (2h27m12.586583431s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:13.277508374 (2h27m12.866853361s) [upload=2577]: Done uploading segments... Aug 7 12:07:13.277529685 (2h27m12.866874672s) [upload=2577]: Upload finished. err: <nil> Aug 7 12:07:59.647444303 (2h27m59.23678929s) [upload=2586]: Starting upload Aug 7 12:07:59.647485382 (2h27m59.236830369s) [upload=2586]: Done starting upload Aug 7 12:07:59.647544936 (2h27m59.236889923s) [upload=2586]: Uploading segments... Aug 7 12:07:59.647582257 (2h27m59.236927234s) [upload=2586]: Got next segment. Inline: true Aug 7 12:07:59.647585894 (2h27m59.236930881s) [upload=2586]: Waiting for error group managing segments... Aug 7 12:07:59.647649967 (2h27m59.236994954s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:59.647654155 (2h27m59.236999142s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:59.647656389 (2h27m59.237001376s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:59.961215120 (2h27m59.550560107s) [upload=2586]: Done uploading segments... Aug 7 12:07:59.961233604 (2h27m59.550578581s) [upload=2586]: Upload finished. err: <nil> Aug 7 12:09:04.474008716 (2h29m4.063353693s) [upload=2606]: Starting upload Aug 7 12:09:04.474054282 (2h29m4.063399269s) [upload=2606]: Done starting upload Aug 7 12:09:04.474117544 (2h29m4.063462531s) [upload=2606]: Uploading segments... Aug 7 12:09:04.474146590 (2h29m4.063491567s) [upload=2606]: Got next segment. Inline: true Aug 7 12:09:04.474149866 (2h29m4.063494853s) [upload=2606]: Waiting for error group managing segments... Aug 7 12:09:04.474185023 (2h29m4.06353001s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:04.474190423 (2h29m4.0635354s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:04.474193168 (2h29m4.063538145s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:04.800005844 (2h29m4.389350831s) [upload=2606]: Done uploading segments... Aug 7 12:09:04.800024600 (2h29m4.389369587s) [upload=2606]: Upload finished. err: <nil> Aug 7 12:09:43.798285096 (2h29m43.387630083s) [upload=2616]: Starting upload Aug 7 12:09:43.798345140 (2h29m43.387690127s) [upload=2616]: Done starting upload Aug 7 12:09:43.798408411 (2h29m43.387753398s) [upload=2616]: Uploading segments... Aug 7 12:09:43.798439370 (2h29m43.387784357s) [upload=2616]: Got next segment. Inline: true Aug 7 12:09:43.798443217 (2h29m43.387788204s) [upload=2616]: Waiting for error group managing segments... Aug 7 12:09:43.798475981 (2h29m43.387820968s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:43.798479978 (2h29m43.387824965s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:43.798482313 (2h29m43.38782729s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:44.143582671 (2h29m43.732927658s) [upload=2616]: Done uploading segments... Aug 7 12:09:44.143598691 (2h29m43.732943668s) [upload=2616]: Upload finished. err: <nil> Aug 7 12:10:27.584576925 (2h30m27.173921912s) [upload=2629]: Starting upload Aug 7 12:10:27.584629195 (2h30m27.173974172s) [upload=2629]: Done starting upload Aug 7 12:10:27.584678159 (2h30m27.174023146s) [upload=2629]: Uploading segments... Aug 7 12:10:27.584707415 (2h30m27.174052402s) [upload=2629]: Got next segment. Inline: true Aug 7 12:10:27.584711312 (2h30m27.174056299s) [upload=2629]: Waiting for error group managing segments... Aug 7 12:10:27.584743203 (2h30m27.17408819s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:10:27.584747040 (2h30m27.174092027s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:10:27.584749435 (2h30m27.174094422s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:10:27.887810742 (2h30m27.477155729s) [upload=2629]: Done uploading segments... Aug 7 12:10:27.887828937 (2h30m27.477173914s) [upload=2629]: Upload finished. err: <nil> Aug 7 12:11:12.432073435 (2h31m12.021418422s) [upload=2642]: Starting upload Aug 7 12:11:12.432118301 (2h31m12.021463278s) [upload=2642]: Done starting upload Aug 7 12:11:12.432168257 (2h31m12.021513244s) [upload=2642]: Uploading segments... Aug 7 12:11:12.432197673 (2h31m12.02154266s) [upload=2642]: Got next segment. Inline: true Aug 7 12:11:12.432201360 (2h31m12.021546347s) [upload=2642]: Waiting for error group managing segments... Aug 7 12:11:12.432235225 (2h31m12.021580212s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:11:12.432238791 (2h31m12.021583778s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:11:12.432241005 (2h31m12.021585992s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:11:12.826402410 (2h31m12.415747397s) [upload=2642]: Done uploading segments... Aug 7 12:11:12.826420154 (2h31m12.415765141s) [upload=2642]: Upload finished. err: <nil> Aug 7 12:12:01.525617822 (2h32m1.114962819s) [upload=2656]: Starting upload Aug 7 12:12:01.525663079 (2h32m1.115008066s) [upload=2656]: Done starting upload Aug 7 12:12:01.525738163 (2h32m1.11508316s) [upload=2656]: Uploading segments... Aug 7 12:12:01.525854065 (2h32m1.115199052s) [upload=2656]: Got next segment. Inline: true Aug 7 12:12:01.525860697 (2h32m1.115205684s) [upload=2656]: Waiting for error group managing segments... Aug 7 12:12:01.525912907 (2h32m1.115257894s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:01.525919960 (2h32m1.115264957s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:01.525923287 (2h32m1.115268284s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:01.969933727 (2h32m1.559278714s) [upload=2656]: Done uploading segments... Aug 7 12:12:01.969966339 (2h32m1.559311326s) [upload=2656]: Upload finished. err: <nil> Aug 7 12:12:23.526243134 (2h32m23.115588181s) [upload=2664]: Starting upload Aug 7 12:12:23.526339669 (2h32m23.115684696s) [upload=2664]: Done starting upload Aug 7 12:12:23.526447936 (2h32m23.115792923s) [upload=2664]: Uploading segments... Aug 7 12:12:23.526522438 (2h32m23.115867445s) [upload=2664]: Got next segment. Inline: true Aug 7 12:12:23.526530534 (2h32m23.115875551s) [upload=2664]: Waiting for error group managing segments... Aug 7 12:12:23.526585439 (2h32m23.115930446s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:23.526592743 (2h32m23.11593775s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:23.526597251 (2h32m23.115942248s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:23.902417816 (2h32m23.491762803s) [upload=2664]: Done uploading segments... Aug 7 12:12:23.902436842 (2h32m23.491781829s) [upload=2664]: Upload finished. err: <nil> Aug 7 12:12:56.180563006 (2h32m55.769907993s) [upload=2668]: Starting upload Aug 7 12:12:56.180605848 (2h32m55.769950835s) [upload=2668]: Done starting upload Aug 7 12:12:56.180665191 (2h32m55.770010178s) [upload=2668]: Uploading segments... Aug 7 12:12:56.180703014 (2h32m55.770048001s) [upload=2668]: Got next segment. Inline: true Aug 7 12:12:56.180706641 (2h32m55.770051618s) [upload=2668]: Waiting for error group managing segments... Aug 7 12:12:56.180740676 (2h32m55.770085663s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:56.180744884 (2h32m55.770089871s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:56.180747298 (2h32m55.770092285s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:56.490488305 (2h32m56.079833292s) [upload=2668]: Done uploading segments... Aug 7 12:12:56.490511128 (2h32m56.079856115s) [upload=2668]: Upload finished. err: <nil> Aug 7 12:13:11.317339795 (2h33m10.906684772s) [upload=2675]: Starting upload Aug 7 12:13:11.317388448 (2h33m10.906733435s) [upload=2675]: Done starting upload Aug 7 12:13:11.317435919 (2h33m10.906780906s) [upload=2675]: Uploading segments... Aug 7 12:13:11.317477428 (2h33m10.906822415s) [upload=2675]: Got next segment. Inline: true Aug 7 12:13:11.317481295 (2h33m10.906826283s) [upload=2675]: Waiting for error group managing segments... Aug 7 12:13:11.317516183 (2h33m10.90686117s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:11.317520180 (2h33m10.906865168s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil>
Duration: 1.966504874s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........X...............................................X..] duration:
1.894624384s
FAIL (2025-08-07 12:14:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after df11e3eae1ac8f408d918f55d929d54d4e5df3d8eaae7e4f8e4c7c8038d1c94c != before bd9ccba63d0a10a25cb9e58291141281665e96854a4357fa64c5bb1ae08d3d6f ======LOGS====== *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:06:53.059666435 (2h26m52.649011472s) [upload=2569]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:06:53.386093536 (2h26m52.975438523s) [upload=2569]: Done uploading segments... Aug 7 12:06:53.386195922 (2h26m52.975540909s) [upload=2569]: Upload finished. err: <nil> Aug 7 12:07:12.996984187 (2h27m12.586329175s) [upload=2577]: Starting upload Aug 7 12:07:12.997029885 (2h27m12.586374872s) [upload=2577]: Done starting upload Aug 7 12:07:12.997091613 (2h27m12.5864366s) [upload=2577]: Uploading segments... Aug 7 12:07:12.997174532 (2h27m12.586519509s) [upload=2577]: Got next segment. Inline: true Aug 7 12:07:12.997179731 (2h27m12.586524719s) [upload=2577]: Waiting for error group managing segments... Aug 7 12:07:12.997230749 (2h27m12.586575726s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:12.997236009 (2h27m12.586580996s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:12.997238444 (2h27m12.586583431s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:13.277508374 (2h27m12.866853361s) [upload=2577]: Done uploading segments... Aug 7 12:07:13.277529685 (2h27m12.866874672s) [upload=2577]: Upload finished. err: <nil> Aug 7 12:07:59.647444303 (2h27m59.23678929s) [upload=2586]: Starting upload Aug 7 12:07:59.647485382 (2h27m59.236830369s) [upload=2586]: Done starting upload Aug 7 12:07:59.647544936 (2h27m59.236889923s) [upload=2586]: Uploading segments... Aug 7 12:07:59.647582257 (2h27m59.236927234s) [upload=2586]: Got next segment. Inline: true Aug 7 12:07:59.647585894 (2h27m59.236930881s) [upload=2586]: Waiting for error group managing segments... Aug 7 12:07:59.647649967 (2h27m59.236994954s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:59.647654155 (2h27m59.236999142s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:59.647656389 (2h27m59.237001376s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:59.961215120 (2h27m59.550560107s) [upload=2586]: Done uploading segments... Aug 7 12:07:59.961233604 (2h27m59.550578581s) [upload=2586]: Upload finished. err: <nil> Aug 7 12:09:04.474008716 (2h29m4.063353693s) [upload=2606]: Starting upload Aug 7 12:09:04.474054282 (2h29m4.063399269s) [upload=2606]: Done starting upload Aug 7 12:09:04.474117544 (2h29m4.063462531s) [upload=2606]: Uploading segments... Aug 7 12:09:04.474146590 (2h29m4.063491567s) [upload=2606]: Got next segment. Inline: true Aug 7 12:09:04.474149866 (2h29m4.063494853s) [upload=2606]: Waiting for error group managing segments... Aug 7 12:09:04.474185023 (2h29m4.06353001s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:04.474190423 (2h29m4.0635354s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:04.474193168 (2h29m4.063538145s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:04.800005844 (2h29m4.389350831s) [upload=2606]: Done uploading segments... Aug 7 12:09:04.800024600 (2h29m4.389369587s) [upload=2606]: Upload finished. err: <nil> Aug 7 12:09:43.798285096 (2h29m43.387630083s) [upload=2616]: Starting upload Aug 7 12:09:43.798345140 (2h29m43.387690127s) [upload=2616]: Done starting upload Aug 7 12:09:43.798408411 (2h29m43.387753398s) [upload=2616]: Uploading segments... Aug 7 12:09:43.798439370 (2h29m43.387784357s) [upload=2616]: Got next segment. Inline: true Aug 7 12:09:43.798443217 (2h29m43.387788204s) [upload=2616]: Waiting for error group managing segments... Aug 7 12:09:43.798475981 (2h29m43.387820968s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:43.798479978 (2h29m43.387824965s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:43.798482313 (2h29m43.38782729s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:44.143582671 (2h29m43.732927658s) [upload=2616]: Done uploading segments... Aug 7 12:09:44.143598691 (2h29m43.732943668s) [upload=2616]: Upload finished. err: <nil> Aug 7 12:10:27.584576925 (2h30m27.173921912s) [upload=2629]: Starting upload Aug 7 12:10:27.584629195 (2h30m27.173974172s) [upload=2629]: Done starting upload Aug 7 12:10:27.584678159 (2h30m27.174023146s) [upload=2629]: Uploading segments... Aug 7 12:10:27.584707415 (2h30m27.174052402s) [upload=2629]: Got next segment. Inline: true Aug 7 12:10:27.584711312 (2h30m27.174056299s) [upload=2629]: Waiting for error group managing segments... Aug 7 12:10:27.584743203 (2h30m27.17408819s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:10:27.584747040 (2h30m27.174092027s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:10:27.584749435 (2h30m27.174094422s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:10:27.887810742 (2h30m27.477155729s) [upload=2629]: Done uploading segments... Aug 7 12:10:27.887828937 (2h30m27.477173914s) [upload=2629]: Upload finished. err: <nil> Aug 7 12:11:12.432073435 (2h31m12.021418422s) [upload=2642]: Starting upload Aug 7 12:11:12.432118301 (2h31m12.021463278s) [upload=2642]: Done starting upload Aug 7 12:11:12.432168257 (2h31m12.021513244s) [upload=2642]: Uploading segments... Aug 7 12:11:12.432197673 (2h31m12.02154266s) [upload=2642]: Got next segment. Inline: true Aug 7 12:11:12.432201360 (2h31m12.021546347s) [upload=2642]: Waiting for error group managing segments... Aug 7 12:11:12.432235225 (2h31m12.021580212s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:11:12.432238791 (2h31m12.021583778s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:11:12.432241005 (2h31m12.021585992s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:11:12.826402410 (2h31m12.415747397s) [upload=2642]: Done uploading segments... Aug 7 12:11:12.826420154 (2h31m12.415765141s) [upload=2642]: Upload finished. err: <nil> Aug 7 12:12:01.525617822 (2h32m1.114962819s) [upload=2656]: Starting upload Aug 7 12:12:01.525663079 (2h32m1.115008066s) [upload=2656]: Done starting upload Aug 7 12:12:01.525738163 (2h32m1.11508316s) [upload=2656]: Uploading segments... Aug 7 12:12:01.525854065 (2h32m1.115199052s) [upload=2656]: Got next segment. Inline: true Aug 7 12:12:01.525860697 (2h32m1.115205684s) [upload=2656]: Waiting for error group managing segments... Aug 7 12:12:01.525912907 (2h32m1.115257894s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:01.525919960 (2h32m1.115264957s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:01.525923287 (2h32m1.115268284s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:01.969933727 (2h32m1.559278714s) [upload=2656]: Done uploading segments... Aug 7 12:12:01.969966339 (2h32m1.559311326s) [upload=2656]: Upload finished. err: <nil> Aug 7 12:12:23.526243134 (2h32m23.115588181s) [upload=2664]: Starting upload Aug 7 12:12:23.526339669 (2h32m23.115684696s) [upload=2664]: Done starting upload Aug 7 12:12:23.526447936 (2h32m23.115792923s) [upload=2664]: Uploading segments... Aug 7 12:12:23.526522438 (2h32m23.115867445s) [upload=2664]: Got next segment. Inline: true Aug 7 12:12:23.526530534 (2h32m23.115875551s) [upload=2664]: Waiting for error group managing segments... Aug 7 12:12:23.526585439 (2h32m23.115930446s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:23.526592743 (2h32m23.11593775s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:23.526597251 (2h32m23.115942248s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:23.902417816 (2h32m23.491762803s) [upload=2664]: Done uploading segments... Aug 7 12:12:23.902436842 (2h32m23.491781829s) [upload=2664]: Upload finished. err: <nil> Aug 7 12:12:56.180563006 (2h32m55.769907993s) [upload=2668]: Starting upload Aug 7 12:12:56.180605848 (2h32m55.769950835s) [upload=2668]: Done starting upload Aug 7 12:12:56.180665191 (2h32m55.770010178s) [upload=2668]: Uploading segments... Aug 7 12:12:56.180703014 (2h32m55.770048001s) [upload=2668]: Got next segment. Inline: true Aug 7 12:12:56.180706641 (2h32m55.770051618s) [upload=2668]: Waiting for error group managing segments... Aug 7 12:12:56.180740676 (2h32m55.770085663s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:56.180744884 (2h32m55.770089871s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:56.180747298 (2h32m55.770092285s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:56.490488305 (2h32m56.079833292s) [upload=2668]: Done uploading segments... Aug 7 12:12:56.490511128 (2h32m56.079856115s) [upload=2668]: Upload finished. err: <nil> Aug 7 12:13:11.317339795 (2h33m10.906684772s) [upload=2675]: Starting upload Aug 7 12:13:11.317388448 (2h33m10.906733435s) [upload=2675]: Done starting upload Aug 7 12:13:11.317435919 (2h33m10.906780906s) [upload=2675]: Uploading segments... Aug 7 12:13:11.317477428 (2h33m10.906822415s) [upload=2675]: Got next segment. Inline: true Aug 7 12:13:11.317481295 (2h33m10.906826283s) [upload=2675]: Waiting for error group managing segments... Aug 7 12:13:11.317516183 (2h33m10.90686117s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:11.317520180 (2h33m10.906865168s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil>
Duration: 1.894647858s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
5.999274011s
FAIL (2025-08-07 12:14:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.999296534s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........X...............................................X.] duration:
1.948122261s
FAIL (2025-08-07 12:13:43 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after df11e3eae1ac8f408d918f55d929d54d4e5df3d8eaae7e4f8e4c7c8038d1c94c != before bd9ccba63d0a10a25cb9e58291141281665e96854a4357fa64c5bb1ae08d3d6f ======LOGS====== *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:06:53.059666435 (2h26m52.649011472s) [upload=2569]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:06:53.386093536 (2h26m52.975438523s) [upload=2569]: Done uploading segments... Aug 7 12:06:53.386195922 (2h26m52.975540909s) [upload=2569]: Upload finished. err: <nil> Aug 7 12:07:12.996984187 (2h27m12.586329175s) [upload=2577]: Starting upload Aug 7 12:07:12.997029885 (2h27m12.586374872s) [upload=2577]: Done starting upload Aug 7 12:07:12.997091613 (2h27m12.5864366s) [upload=2577]: Uploading segments... Aug 7 12:07:12.997174532 (2h27m12.586519509s) [upload=2577]: Got next segment. Inline: true Aug 7 12:07:12.997179731 (2h27m12.586524719s) [upload=2577]: Waiting for error group managing segments... Aug 7 12:07:12.997230749 (2h27m12.586575726s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:12.997236009 (2h27m12.586580996s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:12.997238444 (2h27m12.586583431s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:13.277508374 (2h27m12.866853361s) [upload=2577]: Done uploading segments... Aug 7 12:07:13.277529685 (2h27m12.866874672s) [upload=2577]: Upload finished. err: <nil> Aug 7 12:07:59.647444303 (2h27m59.23678929s) [upload=2586]: Starting upload Aug 7 12:07:59.647485382 (2h27m59.236830369s) [upload=2586]: Done starting upload Aug 7 12:07:59.647544936 (2h27m59.236889923s) [upload=2586]: Uploading segments... Aug 7 12:07:59.647582257 (2h27m59.236927234s) [upload=2586]: Got next segment. Inline: true Aug 7 12:07:59.647585894 (2h27m59.236930881s) [upload=2586]: Waiting for error group managing segments... Aug 7 12:07:59.647649967 (2h27m59.236994954s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:59.647654155 (2h27m59.236999142s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:59.647656389 (2h27m59.237001376s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:59.961215120 (2h27m59.550560107s) [upload=2586]: Done uploading segments... Aug 7 12:07:59.961233604 (2h27m59.550578581s) [upload=2586]: Upload finished. err: <nil> Aug 7 12:09:04.474008716 (2h29m4.063353693s) [upload=2606]: Starting upload Aug 7 12:09:04.474054282 (2h29m4.063399269s) [upload=2606]: Done starting upload Aug 7 12:09:04.474117544 (2h29m4.063462531s) [upload=2606]: Uploading segments... Aug 7 12:09:04.474146590 (2h29m4.063491567s) [upload=2606]: Got next segment. Inline: true Aug 7 12:09:04.474149866 (2h29m4.063494853s) [upload=2606]: Waiting for error group managing segments... Aug 7 12:09:04.474185023 (2h29m4.06353001s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:04.474190423 (2h29m4.0635354s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:04.474193168 (2h29m4.063538145s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:04.800005844 (2h29m4.389350831s) [upload=2606]: Done uploading segments... Aug 7 12:09:04.800024600 (2h29m4.389369587s) [upload=2606]: Upload finished. err: <nil> Aug 7 12:09:43.798285096 (2h29m43.387630083s) [upload=2616]: Starting upload Aug 7 12:09:43.798345140 (2h29m43.387690127s) [upload=2616]: Done starting upload Aug 7 12:09:43.798408411 (2h29m43.387753398s) [upload=2616]: Uploading segments... Aug 7 12:09:43.798439370 (2h29m43.387784357s) [upload=2616]: Got next segment. Inline: true Aug 7 12:09:43.798443217 (2h29m43.387788204s) [upload=2616]: Waiting for error group managing segments... Aug 7 12:09:43.798475981 (2h29m43.387820968s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:43.798479978 (2h29m43.387824965s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:43.798482313 (2h29m43.38782729s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:44.143582671 (2h29m43.732927658s) [upload=2616]: Done uploading segments... Aug 7 12:09:44.143598691 (2h29m43.732943668s) [upload=2616]: Upload finished. err: <nil> Aug 7 12:10:27.584576925 (2h30m27.173921912s) [upload=2629]: Starting upload Aug 7 12:10:27.584629195 (2h30m27.173974172s) [upload=2629]: Done starting upload Aug 7 12:10:27.584678159 (2h30m27.174023146s) [upload=2629]: Uploading segments... Aug 7 12:10:27.584707415 (2h30m27.174052402s) [upload=2629]: Got next segment. Inline: true Aug 7 12:10:27.584711312 (2h30m27.174056299s) [upload=2629]: Waiting for error group managing segments... Aug 7 12:10:27.584743203 (2h30m27.17408819s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:10:27.584747040 (2h30m27.174092027s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:10:27.584749435 (2h30m27.174094422s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:10:27.887810742 (2h30m27.477155729s) [upload=2629]: Done uploading segments... Aug 7 12:10:27.887828937 (2h30m27.477173914s) [upload=2629]: Upload finished. err: <nil> Aug 7 12:11:12.432073435 (2h31m12.021418422s) [upload=2642]: Starting upload Aug 7 12:11:12.432118301 (2h31m12.021463278s) [upload=2642]: Done starting upload Aug 7 12:11:12.432168257 (2h31m12.021513244s) [upload=2642]: Uploading segments... Aug 7 12:11:12.432197673 (2h31m12.02154266s) [upload=2642]: Got next segment. Inline: true Aug 7 12:11:12.432201360 (2h31m12.021546347s) [upload=2642]: Waiting for error group managing segments... Aug 7 12:11:12.432235225 (2h31m12.021580212s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:11:12.432238791 (2h31m12.021583778s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:11:12.432241005 (2h31m12.021585992s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:11:12.826402410 (2h31m12.415747397s) [upload=2642]: Done uploading segments... Aug 7 12:11:12.826420154 (2h31m12.415765141s) [upload=2642]: Upload finished. err: <nil> Aug 7 12:12:01.525617822 (2h32m1.114962819s) [upload=2656]: Starting upload Aug 7 12:12:01.525663079 (2h32m1.115008066s) [upload=2656]: Done starting upload Aug 7 12:12:01.525738163 (2h32m1.11508316s) [upload=2656]: Uploading segments... Aug 7 12:12:01.525854065 (2h32m1.115199052s) [upload=2656]: Got next segment. Inline: true Aug 7 12:12:01.525860697 (2h32m1.115205684s) [upload=2656]: Waiting for error group managing segments... Aug 7 12:12:01.525912907 (2h32m1.115257894s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:01.525919960 (2h32m1.115264957s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:01.525923287 (2h32m1.115268284s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:01.969933727 (2h32m1.559278714s) [upload=2656]: Done uploading segments... Aug 7 12:12:01.969966339 (2h32m1.559311326s) [upload=2656]: Upload finished. err: <nil> Aug 7 12:12:23.526243134 (2h32m23.115588181s) [upload=2664]: Starting upload Aug 7 12:12:23.526339669 (2h32m23.115684696s) [upload=2664]: Done starting upload Aug 7 12:12:23.526447936 (2h32m23.115792923s) [upload=2664]: Uploading segments... Aug 7 12:12:23.526522438 (2h32m23.115867445s) [upload=2664]: Got next segment. Inline: true Aug 7 12:12:23.526530534 (2h32m23.115875551s) [upload=2664]: Waiting for error group managing segments... Aug 7 12:12:23.526585439 (2h32m23.115930446s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:23.526592743 (2h32m23.11593775s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:23.526597251 (2h32m23.115942248s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:23.902417816 (2h32m23.491762803s) [upload=2664]: Done uploading segments... Aug 7 12:12:23.902436842 (2h32m23.491781829s) [upload=2664]: Upload finished. err: <nil> Aug 7 12:12:56.180563006 (2h32m55.769907993s) [upload=2668]: Starting upload Aug 7 12:12:56.180605848 (2h32m55.769950835s) [upload=2668]: Done starting upload Aug 7 12:12:56.180665191 (2h32m55.770010178s) [upload=2668]: Uploading segments... Aug 7 12:12:56.180703014 (2h32m55.770048001s) [upload=2668]: Got next segment. Inline: true Aug 7 12:12:56.180706641 (2h32m55.770051618s) [upload=2668]: Waiting for error group managing segments... Aug 7 12:12:56.180740676 (2h32m55.770085663s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:56.180744884 (2h32m55.770089871s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:56.180747298 (2h32m55.770092285s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:56.490488305 (2h32m56.079833292s) [upload=2668]: Done uploading segments... Aug 7 12:12:56.490511128 (2h32m56.079856115s) [upload=2668]: Upload finished. err: <nil> Aug 7 12:13:11.317339795 (2h33m10.906684772s) [upload=2675]: Starting upload Aug 7 12:13:11.317388448 (2h33m10.906733435s) [upload=2675]: Done starting upload Aug 7 12:13:11.317435919 (2h33m10.906780906s) [upload=2675]: Uploading segments... Aug 7 12:13:11.317477428 (2h33m10.906822415s) [upload=2675]: Got next segment. Inline: true Aug 7 12:13:11.317481295 (2h33m10.906826283s) [upload=2675]: Waiting for error group managing segments... Aug 7 12:13:11.317516183 (2h33m10.90686117s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:11.317520180 (2h33m10.906865168s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil>
Duration: 1.948156217s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
5.678558823s
FAIL (2025-08-07 12:13:29 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.678584272s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...........X...............................................X] duration:
1.613137989s
FAIL (2025-08-07 12:13:12 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after df11e3eae1ac8f408d918f55d929d54d4e5df3d8eaae7e4f8e4c7c8038d1c94c != before bd9ccba63d0a10a25cb9e58291141281665e96854a4357fa64c5bb1ae08d3d6f ======LOGS====== *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:06:53.059666435 (2h26m52.649011472s) [upload=2569]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:06:53.386093536 (2h26m52.975438523s) [upload=2569]: Done uploading segments... Aug 7 12:06:53.386195922 (2h26m52.975540909s) [upload=2569]: Upload finished. err: <nil> Aug 7 12:07:12.996984187 (2h27m12.586329175s) [upload=2577]: Starting upload Aug 7 12:07:12.997029885 (2h27m12.586374872s) [upload=2577]: Done starting upload Aug 7 12:07:12.997091613 (2h27m12.5864366s) [upload=2577]: Uploading segments... Aug 7 12:07:12.997174532 (2h27m12.586519509s) [upload=2577]: Got next segment. Inline: true Aug 7 12:07:12.997179731 (2h27m12.586524719s) [upload=2577]: Waiting for error group managing segments... Aug 7 12:07:12.997230749 (2h27m12.586575726s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:12.997236009 (2h27m12.586580996s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:12.997238444 (2h27m12.586583431s) [upload=2577]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:13.277508374 (2h27m12.866853361s) [upload=2577]: Done uploading segments... Aug 7 12:07:13.277529685 (2h27m12.866874672s) [upload=2577]: Upload finished. err: <nil> Aug 7 12:07:59.647444303 (2h27m59.23678929s) [upload=2586]: Starting upload Aug 7 12:07:59.647485382 (2h27m59.236830369s) [upload=2586]: Done starting upload Aug 7 12:07:59.647544936 (2h27m59.236889923s) [upload=2586]: Uploading segments... Aug 7 12:07:59.647582257 (2h27m59.236927234s) [upload=2586]: Got next segment. Inline: true Aug 7 12:07:59.647585894 (2h27m59.236930881s) [upload=2586]: Waiting for error group managing segments... Aug 7 12:07:59.647649967 (2h27m59.236994954s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:07:59.647654155 (2h27m59.236999142s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:07:59.647656389 (2h27m59.237001376s) [upload=2586]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:07:59.961215120 (2h27m59.550560107s) [upload=2586]: Done uploading segments... Aug 7 12:07:59.961233604 (2h27m59.550578581s) [upload=2586]: Upload finished. err: <nil> Aug 7 12:09:04.474008716 (2h29m4.063353693s) [upload=2606]: Starting upload Aug 7 12:09:04.474054282 (2h29m4.063399269s) [upload=2606]: Done starting upload Aug 7 12:09:04.474117544 (2h29m4.063462531s) [upload=2606]: Uploading segments... Aug 7 12:09:04.474146590 (2h29m4.063491567s) [upload=2606]: Got next segment. Inline: true Aug 7 12:09:04.474149866 (2h29m4.063494853s) [upload=2606]: Waiting for error group managing segments... Aug 7 12:09:04.474185023 (2h29m4.06353001s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:04.474190423 (2h29m4.0635354s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:04.474193168 (2h29m4.063538145s) [upload=2606]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:04.800005844 (2h29m4.389350831s) [upload=2606]: Done uploading segments... Aug 7 12:09:04.800024600 (2h29m4.389369587s) [upload=2606]: Upload finished. err: <nil> Aug 7 12:09:43.798285096 (2h29m43.387630083s) [upload=2616]: Starting upload Aug 7 12:09:43.798345140 (2h29m43.387690127s) [upload=2616]: Done starting upload Aug 7 12:09:43.798408411 (2h29m43.387753398s) [upload=2616]: Uploading segments... Aug 7 12:09:43.798439370 (2h29m43.387784357s) [upload=2616]: Got next segment. Inline: true Aug 7 12:09:43.798443217 (2h29m43.387788204s) [upload=2616]: Waiting for error group managing segments... Aug 7 12:09:43.798475981 (2h29m43.387820968s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:09:43.798479978 (2h29m43.387824965s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:09:43.798482313 (2h29m43.38782729s) [upload=2616]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:09:44.143582671 (2h29m43.732927658s) [upload=2616]: Done uploading segments... Aug 7 12:09:44.143598691 (2h29m43.732943668s) [upload=2616]: Upload finished. err: <nil> Aug 7 12:10:27.584576925 (2h30m27.173921912s) [upload=2629]: Starting upload Aug 7 12:10:27.584629195 (2h30m27.173974172s) [upload=2629]: Done starting upload Aug 7 12:10:27.584678159 (2h30m27.174023146s) [upload=2629]: Uploading segments... Aug 7 12:10:27.584707415 (2h30m27.174052402s) [upload=2629]: Got next segment. Inline: true Aug 7 12:10:27.584711312 (2h30m27.174056299s) [upload=2629]: Waiting for error group managing segments... Aug 7 12:10:27.584743203 (2h30m27.17408819s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:10:27.584747040 (2h30m27.174092027s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:10:27.584749435 (2h30m27.174094422s) [upload=2629]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:10:27.887810742 (2h30m27.477155729s) [upload=2629]: Done uploading segments... Aug 7 12:10:27.887828937 (2h30m27.477173914s) [upload=2629]: Upload finished. err: <nil> Aug 7 12:11:12.432073435 (2h31m12.021418422s) [upload=2642]: Starting upload Aug 7 12:11:12.432118301 (2h31m12.021463278s) [upload=2642]: Done starting upload Aug 7 12:11:12.432168257 (2h31m12.021513244s) [upload=2642]: Uploading segments... Aug 7 12:11:12.432197673 (2h31m12.02154266s) [upload=2642]: Got next segment. Inline: true Aug 7 12:11:12.432201360 (2h31m12.021546347s) [upload=2642]: Waiting for error group managing segments... Aug 7 12:11:12.432235225 (2h31m12.021580212s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:11:12.432238791 (2h31m12.021583778s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:11:12.432241005 (2h31m12.021585992s) [upload=2642]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:11:12.826402410 (2h31m12.415747397s) [upload=2642]: Done uploading segments... Aug 7 12:11:12.826420154 (2h31m12.415765141s) [upload=2642]: Upload finished. err: <nil> Aug 7 12:12:01.525617822 (2h32m1.114962819s) [upload=2656]: Starting upload Aug 7 12:12:01.525663079 (2h32m1.115008066s) [upload=2656]: Done starting upload Aug 7 12:12:01.525738163 (2h32m1.11508316s) [upload=2656]: Uploading segments... Aug 7 12:12:01.525854065 (2h32m1.115199052s) [upload=2656]: Got next segment. Inline: true Aug 7 12:12:01.525860697 (2h32m1.115205684s) [upload=2656]: Waiting for error group managing segments... Aug 7 12:12:01.525912907 (2h32m1.115257894s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:01.525919960 (2h32m1.115264957s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:01.525923287 (2h32m1.115268284s) [upload=2656]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:01.969933727 (2h32m1.559278714s) [upload=2656]: Done uploading segments... Aug 7 12:12:01.969966339 (2h32m1.559311326s) [upload=2656]: Upload finished. err: <nil> Aug 7 12:12:23.526243134 (2h32m23.115588181s) [upload=2664]: Starting upload Aug 7 12:12:23.526339669 (2h32m23.115684696s) [upload=2664]: Done starting upload Aug 7 12:12:23.526447936 (2h32m23.115792923s) [upload=2664]: Uploading segments... Aug 7 12:12:23.526522438 (2h32m23.115867445s) [upload=2664]: Got next segment. Inline: true Aug 7 12:12:23.526530534 (2h32m23.115875551s) [upload=2664]: Waiting for error group managing segments... Aug 7 12:12:23.526585439 (2h32m23.115930446s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:23.526592743 (2h32m23.11593775s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:23.526597251 (2h32m23.115942248s) [upload=2664]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:23.902417816 (2h32m23.491762803s) [upload=2664]: Done uploading segments... Aug 7 12:12:23.902436842 (2h32m23.491781829s) [upload=2664]: Upload finished. err: <nil> Aug 7 12:12:56.180563006 (2h32m55.769907993s) [upload=2668]: Starting upload Aug 7 12:12:56.180605848 (2h32m55.769950835s) [upload=2668]: Done starting upload Aug 7 12:12:56.180665191 (2h32m55.770010178s) [upload=2668]: Uploading segments... Aug 7 12:12:56.180703014 (2h32m55.770048001s) [upload=2668]: Got next segment. Inline: true Aug 7 12:12:56.180706641 (2h32m55.770051618s) [upload=2668]: Waiting for error group managing segments... Aug 7 12:12:56.180740676 (2h32m55.770085663s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:12:56.180744884 (2h32m55.770089871s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:12:56.180747298 (2h32m55.770092285s) [upload=2668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:12:56.490488305 (2h32m56.079833292s) [upload=2668]: Done uploading segments... Aug 7 12:12:56.490511128 (2h32m56.079856115s) [upload=2668]: Upload finished. err: <nil> Aug 7 12:13:11.317339795 (2h33m10.906684772s) [upload=2675]: Starting upload Aug 7 12:13:11.317388448 (2h33m10.906733435s) [upload=2675]: Done starting upload Aug 7 12:13:11.317435919 (2h33m10.906780906s) [upload=2675]: Uploading segments... Aug 7 12:13:11.317477428 (2h33m10.906822415s) [upload=2675]: Got next segment. Inline: true Aug 7 12:13:11.317481295 (2h33m10.906826283s) [upload=2675]: Waiting for error group managing segments... Aug 7 12:13:11.317516183 (2h33m10.90686117s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 12:13:11.317520180 (2h33m10.906865168s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 12:13:11.317522485 (2h33m10.906867472s) [upload=2675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 12:13:11.591160497 (2h33m11.180505484s) [upload=2675]: Done uploading segments... Aug 7 12:13:11.591178821 (2h33m11.180523808s) [upload=2675]: Upload finished. err: <nil>
Duration: 1.613167617s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
5.952514139s
FAIL (2025-08-07 12:13:04 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.952541772s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.008761441s
FAIL (2025-08-07 12:12:57 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.008785308s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.887625748s
FAIL (2025-08-07 12:12:41 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.887648031s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
1.999554907s
FAIL (2025-08-07 12:12:24 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.999579924s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.25048942s
FAIL (2025-08-07 12:12:03 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.250514006s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.733262286s
FAIL (2025-08-07 12:11:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.733286413s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
2.008126695s
FAIL (2025-08-07 12:11:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.008153748s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.864870425s
FAIL (2025-08-07 12:10:47 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.864894121s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
1.885681516s
FAIL (2025-08-07 12:10:28 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.885706222s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
1.961483635s
FAIL (2025-08-07 12:09:45 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.961511279s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
5.866862676s
FAIL (2025-08-07 12:09:39 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.866880941s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
1.994175554s
FAIL (2025-08-07 12:09:05 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.994212905s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.658883784s
FAIL (2025-08-07 12:08:36 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.658913331s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
2.020102047s
FAIL (2025-08-07 12:08:00 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.020128598s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.62514178s
FAIL (2025-08-07 12:07:35 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.625167689s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|