Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
1.927761085s
FAIL (2025-08-07 12:07:14 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.927790651s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.008642051s
FAIL (2025-08-07 12:06:54 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.008676576s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
5.949981555s
FAIL (2025-08-07 12:06:32 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.950006904s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.21885651s
FAIL (2025-08-07 12:06:01 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.218915593s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.220155086s
FAIL (2025-08-07 12:05:40 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.220186736s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [X.....................X.....................................] duration:
6.354836974s
FAIL (2025-08-07 12:05:24 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.354862554s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.147849446s
FAIL (2025-08-07 12:05:04 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.147871929s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X.....................X....................................] duration:
5.827767075s
FAIL (2025-08-07 12:04:40 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.827790919s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
1.96553523s
FAIL (2025-08-07 12:04:11 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.965554677s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X.....................X...................................] duration:
6.162866063s
FAIL (2025-08-07 12:04:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.162885222s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X.....................X..................................] duration:
5.810204859s
FAIL (2025-08-07 12:03:29 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.810230607s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
1.872795434s
FAIL (2025-08-07 12:03:26 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.872818056s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
1.850276519s
FAIL (2025-08-07 12:03:06 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.850302881s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X.....................X.................................] duration:
5.865417216s
FAIL (2025-08-07 12:02:37 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.865444148s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.859324469s
FAIL (2025-08-07 12:02:02 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.859350619s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....X.....................X................................] duration:
5.867287934s
FAIL (2025-08-07 12:01:24 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.867313893s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
1.902698825s
FAIL (2025-08-07 12:00:56 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.902734533s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.052055787s
FAIL (2025-08-07 12:00:28 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.052082569s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [......X.....................X...............................] duration:
5.856427557s
FAIL (2025-08-07 12:00:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.856452015s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.278635895s
FAIL (2025-08-07 12:00:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.278666904s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......X.....................X..............................] duration:
5.67810388s
FAIL (2025-08-07 11:59:32 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.678128698s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [X...............................X...........................] duration:
2.182656008s
FAIL (2025-08-07 11:59:14 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.182679875s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.X...............................X..........................] duration:
1.869782337s
FAIL (2025-08-07 11:58:31 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.86980432s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [........X.....................X.............................] duration:
5.848076797s
FAIL (2025-08-07 11:58:21 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.848101264s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..X...............................X.........................] duration:
1.882358519s
FAIL (2025-08-07 11:58:06 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.882383617s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........X.....................X............................] duration:
5.837802407s
FAIL (2025-08-07 11:57:35 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.837827126s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...X...............................X........................] duration:
1.869246523s
FAIL (2025-08-07 11:57:32 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.86927591s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....X...............................X.......................] duration:
1.856190564s
FAIL (2025-08-07 11:57:01 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.856217917s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........X.....................X...........................] duration:
5.82934516s
FAIL (2025-08-07 11:56:47 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.829362734s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X.....................X..........................] duration:
5.834570447s
FAIL (2025-08-07 11:56:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.834596077s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [X....X...............................X......................] duration:
1.858270782s
FAIL (2025-08-07 11:55:48 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.858299798s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [............X.....................X.........................] duration:
5.765349644s
FAIL (2025-08-07 11:55:42 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.7653732s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............X.....................X........................] duration:
5.734274936s
FAIL (2025-08-07 11:55:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.734301157s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.X....X...............................X.....................] duration:
1.843771341s
FAIL (2025-08-07 11:55:01 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.843799876s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..X....X...............................X....................] duration:
1.935819434s
FAIL (2025-08-07 11:54:22 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.935873789s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............X.....................X.......................] duration:
5.952933504s
FAIL (2025-08-07 11:54:14 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.952957141s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...X....X...............................X...................] duration:
2.19918851s
FAIL (2025-08-07 11:53:18 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.199217496s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............X.....................X......................] duration:
5.852818949s
FAIL (2025-08-07 11:53:00 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.852843196s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [....X....X...............................X..................] duration:
1.827838499s
FAIL (2025-08-07 11:52:43 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.827863167s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [................X.....................X.....................] duration:
5.68799409s
FAIL (2025-08-07 11:52:24 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.688022133s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.....X....X...............................X.................] duration:
1.878604146s
FAIL (2025-08-07 11:51:51 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.878637691s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................X.....................X....................] duration:
5.605725688s
FAIL (2025-08-07 11:51:36 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.605743612s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [......X....X...............................X................] duration:
2.027574648s
FAIL (2025-08-07 11:51:35 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.027599595s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................X.....................X...................] duration:
5.905546421s
FAIL (2025-08-07 11:50:57 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.905570647s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.......X....X...............................X...............] duration:
2.422145778s
FAIL (2025-08-07 11:50:46 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.422187999s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................X.....................X..................] duration:
6.620025333s
FAIL (2025-08-07 11:50:35 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.620045373s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [........X....X...............................X..............] duration:
1.847702875s
FAIL (2025-08-07 11:50:06 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.847727672s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.........X....X...............................X.............] duration:
1.866739616s
FAIL (2025-08-07 11:49:30 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== estItem_SegmentMakeInline Aug 7 11:31:15.326529397 (1h51m14.915874384s) [upload=1945]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:31:15.696887103 (1h51m15.28623209s) [upload=1945]: Done uploading segments... Aug 7 11:31:15.696903574 (1h51m15.286248561s) [upload=1945]: Upload finished. err: <nil> Aug 7 11:32:06.590861675 (1h52m6.180206662s) [upload=1961]: Starting upload Aug 7 11:32:06.590908414 (1h52m6.180253401s) [upload=1961]: Done starting upload Aug 7 11:32:06.590958570 (1h52m6.180303547s) [upload=1961]: Uploading segments... Aug 7 11:32:06.590991182 (1h52m6.180336169s) [upload=1961]: Got next segment. Inline: true Aug 7 11:32:06.590995240 (1h52m6.180340227s) [upload=1961]: Waiting for error group managing segments... Aug 7 11:32:06.591027112 (1h52m6.180372209s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:06.591030719 (1h52m6.180375706s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:06.591032903 (1h52m6.18037788s) [upload=1961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:06.950133689 (1h52m6.539478696s) [upload=1961]: Done uploading segments... Aug 7 11:32:06.950156353 (1h52m6.53950134s) [upload=1961]: Upload finished. err: <nil> Aug 7 11:32:49.940109658 (1h52m49.529454645s) [upload=1975]: Starting upload Aug 7 11:32:49.940204599 (1h52m49.529549586s) [upload=1975]: Done starting upload Aug 7 11:32:49.940261788 (1h52m49.529606775s) [upload=1975]: Uploading segments... Aug 7 11:32:49.940291917 (1h52m49.529636904s) [upload=1975]: Got next segment. Inline: true Aug 7 11:32:49.940295474 (1h52m49.529640461s) [upload=1975]: Waiting for error group managing segments... Aug 7 11:32:49.940345960 (1h52m49.529690947s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:32:49.940350809 (1h52m49.529695797s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:32:49.940374044 (1h52m49.529719021s) [upload=1975]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:32:50.215652877 (1h52m49.804997864s) [upload=1975]: Done uploading segments... Aug 7 11:32:50.215669108 (1h52m49.805014095s) [upload=1975]: Upload finished. err: <nil> Aug 7 11:33:54.721039935 (1h53m54.310384952s) [upload=1990]: Starting upload Aug 7 11:33:54.721083028 (1h53m54.310428015s) [upload=1990]: Done starting upload Aug 7 11:33:54.721134537 (1h53m54.310479524s) [upload=1990]: Uploading segments... Aug 7 11:33:54.721164384 (1h53m54.310509371s) [upload=1990]: Got next segment. Inline: true Aug 7 11:33:54.721167991 (1h53m54.310512978s) [upload=1990]: Waiting for error group managing segments... Aug 7 11:33:54.721199912 (1h53m54.310544899s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:33:54.721203448 (1h53m54.310548435s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:33:54.721205823 (1h53m54.3105508s) [upload=1990]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:33:55.004429376 (1h53m54.593774383s) [upload=1990]: Done uploading segments... Aug 7 11:33:55.004452661 (1h53m54.593797658s) [upload=1990]: Upload finished. err: <nil> Aug 7 11:34:46.370172530 (1h54m45.959517517s) [upload=2007]: Starting upload Aug 7 11:34:46.370233286 (1h54m45.959578273s) [upload=2007]: Done starting upload Aug 7 11:34:46.370294062 (1h54m45.959639039s) [upload=2007]: Uploading segments... Aug 7 11:34:46.370323438 (1h54m45.959668415s) [upload=2007]: Got next segment. Inline: true Aug 7 11:34:46.370327265 (1h54m45.959672242s) [upload=2007]: Waiting for error group managing segments... Aug 7 11:34:46.370362162 (1h54m45.959707149s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:34:46.370366481 (1h54m45.959711468s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:34:46.370370799 (1h54m45.959715786s) [upload=2007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:34:46.761516618 (1h54m46.350861605s) [upload=2007]: Done uploading segments... Aug 7 11:34:46.761533399 (1h54m46.350878386s) [upload=2007]: Upload finished. err: <nil> Aug 7 11:35:23.768011839 (1h55m23.357356826s) [upload=2021]: Starting upload Aug 7 11:35:23.768051905 (1h55m23.357396882s) [upload=2021]: Done starting upload Aug 7 11:35:23.768100728 (1h55m23.357445716s) [upload=2021]: Uploading segments... Aug 7 11:35:23.768132389 (1h55m23.357477376s) [upload=2021]: Got next segment. Inline: true Aug 7 11:35:23.768136027 (1h55m23.357481014s) [upload=2021]: Waiting for error group managing segments... Aug 7 11:35:23.768181323 (1h55m23.35752631s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:35:23.768185070 (1h55m23.357530057s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:35:23.768187374 (1h55m23.357532361s) [upload=2021]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:35:24.053511939 (1h55m23.642856926s) [upload=2021]: Done uploading segments... Aug 7 11:35:24.053540133 (1h55m23.64288512s) [upload=2021]: Upload finished. err: <nil> Aug 7 11:36:18.015760173 (1h56m17.60510516s) [upload=2034]: Starting upload Aug 7 11:36:18.015807003 (1h56m17.60515199s) [upload=2034]: Done starting upload Aug 7 11:36:18.015852510 (1h56m17.605197497s) [upload=2034]: Uploading segments... Aug 7 11:36:18.015881315 (1h56m17.605226302s) [upload=2034]: Got next segment. Inline: true Aug 7 11:36:18.015885353 (1h56m17.60523034s) [upload=2034]: Waiting for error group managing segments... Aug 7 11:36:18.015917405 (1h56m17.605262392s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:18.015920981 (1h56m17.605265958s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:18.015923286 (1h56m17.605268273s) [upload=2034]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:18.296456873 (1h56m17.88580185s) [upload=2034]: Done uploading segments... Aug 7 11:36:18.296474226 (1h56m17.885819214s) [upload=2034]: Upload finished. err: <nil> Aug 7 11:36:37.677683434 (1h56m37.267028421s) [upload=2042]: Starting upload Aug 7 11:36:37.677726357 (1h56m37.267071344s) [upload=2042]: Done starting upload Aug 7 11:36:37.677771944 (1h56m37.267116921s) [upload=2042]: Uploading segments... Aug 7 11:36:37.677827260 (1h56m37.267172257s) [upload=2042]: Got next segment. Inline: true Aug 7 11:36:37.677869100 (1h56m37.267214077s) [upload=2042]: Waiting for error group managing segments... Aug 7 11:36:37.677902864 (1h56m37.267247851s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:36:37.677906522 (1h56m37.267251509s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:36:37.677908766 (1h56m37.267253753s) [upload=2042]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:36:37.959295135 (1h56m37.548640122s) [upload=2042]: Done uploading segments... Aug 7 11:36:37.959312338 (1h56m37.548657325s) [upload=2042]: Upload finished. err: <nil> Aug 7 11:37:06.286481710 (1h57m5.875826727s) [upload=2050]: Starting upload Aug 7 11:37:06.286576201 (1h57m5.875921189s) [upload=2050]: Done starting upload Aug 7 11:37:06.286671694 (1h57m5.876016721s) [upload=2050]: Uploading segments... Aug 7 11:37:06.286725527 (1h57m5.876070544s) [upload=2050]: Got next segment. Inline: true Aug 7 11:37:06.286732840 (1h57m5.876077878s) [upload=2050]: Waiting for error group managing segments... Aug 7 11:37:06.286792074 (1h57m5.876137111s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:37:06.286799318 (1h57m5.876144335s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:37:06.286805299 (1h57m5.876150327s) [upload=2050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:37:06.636895255 (1h57m6.226240242s) [upload=2050]: Done uploading segments... Aug 7 11:37:06.636912818 (1h57m6.226257805s) [upload=2050]: Upload finished. err: <nil> Aug 7 11:38:04.191265450 (1h58m3.780610427s) [upload=2066]: Starting upload Aug 7 11:38:04.191309524 (1h58m3.780654511s) [upload=2066]: Done starting upload Aug 7 11:38:04.191355172 (1h58m3.780700149s) [upload=2066]: Uploading segments... Aug 7 11:38:04.191391110 (1h58m3.780736097s) [upload=2066]: Got next segment. Inline: true Aug 7 11:38:04.191395679 (1h58m3.780740666s) [upload=2066]: Waiting for error group managing segments... Aug 7 11:38:04.191433041 (1h58m3.780778028s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:04.191437199 (1h58m3.780782186s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:04.191439724 (1h58m3.780784711s) [upload=2066]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:04.479188966 (1h58m4.068533953s) [upload=2066]: Done uploading segments... Aug 7 11:38:04.479206590 (1h58m4.068551577s) [upload=2066]: Upload finished. err: <nil> Aug 7 11:38:37.706869642 (1h58m37.29621463s) [upload=2076]: Starting upload Aug 7 11:38:37.706915861 (1h58m37.296260848s) [upload=2076]: Done starting upload Aug 7 11:38:37.706984472 (1h58m37.296329459s) [upload=2076]: Uploading segments... Aug 7 11:38:37.707105224 (1h58m37.296450211s) [upload=2076]: Got next segment. Inline: true Aug 7 11:38:37.707110393 (1h58m37.29645537s) [upload=2076]: Waiting for error group managing segments... Aug 7 11:38:37.707156372 (1h58m37.296501359s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 11:38:37.707161642 (1h58m37.296506629s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 11:38:37.707164687 (1h58m37.296509674s) [upload=2076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 11:38:37.928261085 (1h58m37.517606072s) [upload=2076]: Done uploading segments... Aug 7 11:38:37.928284138 (1h58m37.517629165s) [upload=2076]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.866767699s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................X.....................X.................] duration:
5.867296088s
FAIL (2025-08-07 11:49:25 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.867324063s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [X....................X.....................X................] duration:
5.594634995s
FAIL (2025-08-07 11:49:05 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Done begin upload segment.
Aug 7 11:35:14.626321909 (1h55m14.215666896s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626359061 (1h55m14.215704048s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626706635 (1h55m14.216051622s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.626816906 (1h55m14.216161893s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627074369 (1h55m14.216419356s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627353122 (1h55m14.21669811s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627628289 (1h55m14.216973276s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.627966125 (1h55m14.217311112s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628315885 (1h55m14.217660872s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628614406 (1h55m14.217959393s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.628859405 (1h55m14.218204382s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629084796 (1h55m14.218429783s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629309205 (1h55m14.218654192s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629597407 (1h55m14.218942394s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.629856743 (1h55m14.21920172s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630070953 (1h55m14.21941594s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630286876 (1h55m14.219631853s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:14.630496256 (1h55m14.219841233s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Uploading piece...
Aug 7 11:35:15.727038157 (1h55m15.316383144s) [upload=2017]: Next returned nil segment
Aug 7 11:35:15.727061862 (1h55m15.316406879s) [upload=2017]: Waiting for error group managing segments...
Aug 7 11:35:16.004111794 (1h55m15.593456781s) [noise=true , address=193.149.164.143:4702 , piece_id=T7V6RYVYOITF2F5NZ63G6JHZWMHWC4X3RZKMKDEHCYKKPFPUX4PA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.069572358 (1h55m15.658917345s) [noise=true , address=64.49.13.140:4514 , piece_id=JCXDJHISZDGZVIIZFWRFWPJUVXVTSSIJWFAY4U445BHA27MVRHGA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.532790871 (1h55m16.122135878s) [noise=true , address=104.36.82.34:4565 , piece_id=B3RWES2FA2NCGVYM74YKTDEEKA3WJLLA7CZO7W7QEI4TX64Q4DOQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.555450445 (1h55m16.144795452s) [noise=true , address=104.36.82.37:4666 , piece_id=SBP2WMR2MV4YY7RPKJDG5WHSE6OQQOWMM2GHUCDWHSIMUHWIW2KA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.572038993 (1h55m16.16138398s) [noise=true , address=104.36.82.52:4108 , piece_id=JAO56FNLWVV37O5EIGF2S2NSBHD6GFLWLYIUKR6EIPZR3DWVFGNQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.605601177 (1h55m16.194946164s) [noise=true , address=104.36.82.62:4413 , piece_id=UP2LMBVXQRPFFU4JXLVBJRFLHTFJE4OOERT4WTQUTZSRPLREXTIA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.609587304 (1h55m16.198932291s) [noise=true , address=45.91.116.11:4151 , piece_id=ETPDKNCJFKCEVP4OHVPSDLK2757QIYG7TEQ47GEYVLL3AHSAUWHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.615805481 (1h55m16.205150468s) [noise=true , address=104.36.82.43:4865 , piece_id=BXPYJHJFXA6UJCP2AMYFAYIE3VNN7RUEWMS6HHKCV6RJ72WK2QXA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620149773 (1h55m16.20949477s) [noise=true , address=104.36.82.23:4458 , piece_id=3BFQYMUZ3LPUDWD25NJJBDED3QBQGYXJSDO4YEJ4P7FIKJ45BOBA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.620780179 (1h55m16.210125186s) [noise=true , address=104.36.82.9:4036 , piece_id=4ZCOD6JT55WNCEDILL3VYQRV5SNCOKDR5LCX2MCSLBWY2AHEXIEA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.632133931 (1h55m16.221478909s) [noise=true , address=104.36.82.59:4301 , piece_id=DZG3L5JDRULGT2UZGKCPDBMX6LLZCZG443DAJ3NZ5OI6UDEVYEWQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.647615953 (1h55m16.23696094s) [noise=true , address=104.36.82.5:4847 , piece_id=JOA7BP536NV44CLQNVL4YGV3ZB4RVSBDLVNWLMW2OFBY77CX2RAA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.650694113 (1h55m16.2400391s) [noise=true , address=104.36.82.44:4896 , piece_id=AGSCFUXCLVEV5I7QT3JARXU2OUM5YF2H7BBB7EAAJ6GRRBFPW7LA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704161239 (1h55m16.293506226s) [noise=true , address=104.36.82.61:4403 , piece_id=B4HFGDP4KU2E4AGADKP4QUJXOLSEWS2UNND34XKDY6AOXNIUAVXQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: <nil>
Aug 7 11:35:16.704187649 (1h55m16.293532636s) [seg_pos={0 0}, upload=2017]: Segment reached optimal threshold of 14 pieces.
Aug 7 11:35:16.704305635 (1h55m16.293650632s) [noise=true , address=104.36.82.16:4327 , piece_id=C33A4VYWP7P5U3UP6LYGJZSYJ3N5XJ2ZKF74BGQOTPNS3XAROSHQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704355150 (1h55m16.293700137s) [noise=true , address=104.36.82.46:4490 , piece_id=5IVAOS7CKTK4IZ2DL7WDNWRGG44Y3KDFC2UGAQF7NEX365UDO6QQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704407951 (1h55m16.293752938s) [noise=true , address=45.91.116.4:4954 , piece_id=CIA5657CCZ2T7M3EAI3HR76J62HO5BBIQAWXMLNHSNXRAGOBTFQQ, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tT1yhzA7AUuwesNf62NHyVxgSvbrmFzchYSmYdqihRjuAgvYa): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704554781 (1h55m16.293899779s) [noise=true , address=104.36.82.8:4993 , piece_id=YGCL3LUQUV4NR3BGWPWI5ZYJLOQLJYIEURQYHEZNP6TRDIGFACUA, seg_pos={0 0}, upload=2017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 7 11:35:16.704626058 (1h55m16.293971065s) [seg_pos={0 0}, upload=2017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 11:35:16.704725348 (1h55m16.294070376s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 11:35:16.704735698 (1h55m16.294080705s) [upload=2017]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 11:35:17.014937397 (1h55m16.604282384s) [upload=2017]: Done uploading segments...
Aug 7 11:35:17.014955211 (1h55m16.604300198s) [upload=2017]: Upload finished. err: <nil>
Aug 7 11:35:56.153530354 (1h55m55.742875341s) [upload=2029]: Starting upload
Aug 7 11:35:56.153622601 (1h55m55.742967588s) [upload=2029]: Done starting upload
Aug 7 11:35:56.153791894 (1h55m55.743136881s) [upload=2029]: Uploading segments...
Aug 7 11:35:56.153967210 (1h55m55.743312197s) [upload=2029]: Got next segment. Inline: false
Aug 7 11:35:56.154043916 (1h55m55.743388903s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 11:35:56.154058605 (1h55m55.743403592s) [upload=2029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 11:35:56.368151533 (1h55m55.95749652s) [upload=2029]: Next segment err: metaclient: Too Many Requests
Aug 7 11:35:56.368174026 (1h55m55.957519023s) [upload=2029]: Done uploading segments...
Aug 7 11:35:56.368194535 (1h55m55.957539522s) [upload=2029]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.594659903s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|