Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.002154481s
FAIL (2025-07-19 06:57:17 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
ARML7R2U7MK5UBOQLTMNEEL2KMPQ, seg_pos={0 0}, upload=6394]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 19 06:56:01.174242029 (8h5m7.546821384s) [noise=true , address=104.36.82.10:4079 , piece_id=CQZTVCO3IEWAGADQCO7PUGO3ZYPMQXKHZ2UPR4X3WXR4UOXD23DQ, seg_pos={0 0}, upload=6394]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF
Jul 19 06:56:01.174255615 (8h5m7.54683497s) [seg_pos={0 0}, upload=6394]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 19 06:56:01.174325238 (8h5m7.546904634s) [upload=6394]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 19 06:56:01.174333464 (8h5m7.546912819s) [upload=6394]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 19 06:56:01.474962706 (8h5m7.847542061s) [upload=6394]: Done uploading segments...
Jul 19 06:56:01.474982223 (8h5m7.847561578s) [upload=6394]: Upload finished. err: <nil>
Jul 19 06:56:48.315803186 (8h5m54.688382541s) [upload=6403]: Starting upload
Jul 19 06:56:48.315875724 (8h5m54.688455079s) [upload=6403]: Done starting upload
Jul 19 06:56:48.315947812 (8h5m54.688527167s) [upload=6403]: Uploading segments...
Jul 19 06:56:48.316122416 (8h5m54.688701761s) [upload=6403]: Got next segment. Inline: false
Jul 19 06:56:48.316150149 (8h5m54.688729494s) [upload=6403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 19 06:56:48.316155278 (8h5m54.688734633s) [upload=6403]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 19 06:56:48.549320531 (8h5m54.921899886s) [seg_pos={0 0}, upload=6403]: Begin upload segment...
Jul 19 06:56:48.549361600 (8h5m54.921940955s) [seg_pos={0 0}, upload=6403]: Done begin upload segment.
Jul 19 06:56:48.549382059 (8h5m54.921961404s) [noise=true , address=77.90.31.21:4877 , piece_id=EDLQFKGHOTHTLWL7PPUOMRTA6JIL7TF7HU22JNO2EVVXETFLUHYQ, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.549433146 (8h5m54.922012501s) [noise=true , address=104.36.82.13:4185 , piece_id=JHS6G7M7N5IVY2VPSPHJPVRZQHQJIYBU54KKUW47COZP2YGQ2XGA, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.549694476 (8h5m54.922273831s) [noise=true , address=45.91.116.5:4299 , piece_id=OHEAHX6PK3JW6K3BYMJQYK4HDNP6K7DEJOW7EOQKK2TC4P6NW4CA, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.549875533 (8h5m54.922454888s) [noise=true , address=45.91.116.13:4153 , piece_id=MPAR7SWMVQ236Z76HU7FC3LMQ4NRUD3TN64HT3MQRWTWGBQ6YDSQ, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.549937652 (8h5m54.922517007s) [noise=true , address=45.91.116.8:4045 , piece_id=G5QHS6Y4T2JY7PYOXKUGSZM75F4NWF37L45H434N5WOZKEDUMPWA, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.550155920 (8h5m54.922735275s) [noise=true , address=45.91.116.11:4117 , piece_id=BZQFV37K7LE4Z2K54HPA6PBIVNZEC6SBX4HYUE6YGRCHLYUOOEBA, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.550269487 (8h5m54.922848832s) [noise=true , address=45.91.116.12:4227 , piece_id=A25PGAY3W24PI2WFW6NYMWVSW3NQK4DC5ZJMC4VISWYQICT3SJQA, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.550430595 (8h5m54.92300994s) [noise=true , address=104.36.82.15:4259 , piece_id=HAQGLMVTGJXXK334AOVADV45LX4MBWBSDOUHZ7HXYB4CZJ6SWM5A, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.550495319 (8h5m54.923074664s) [noise=true , address=77.90.31.19:4841 , piece_id=WJMRBXWLIAPBM4YW6SZDQM3A536PDDRMOJF5R4J2LVRDVJPULP4A, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.550699370 (8h5m54.923278715s) [noise=true , address=45.91.116.4:4938 , piece_id=ZIJIIBKIGQY2BRI3DVAEWWLPPPUDB2RYMYV2EAD3EZLO2K2P2WTA, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.550722122 (8h5m54.923301477s) [noise=true , address=77.90.31.15:4698 , piece_id=6PREKXY6754B6THCWIZRYR3AO3S2XGPR4GSI7WQ2UNJW2NDWAYEQ, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.550951211 (8h5m54.923530566s) [noise=true , address=77.90.31.10:4520 , piece_id=PV2J4IO3YYWYVEKXTS4C2CT3BIFNDJNJOYCYDQC5LQC2B4TZCU3A, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.550955099 (8h5m54.923534444s) [noise=true , address=104.36.82.2:4731 , piece_id=MIOJJFTFCRXMSFE4FU2AKBFTPVIX5JEGOJV5FIX5OGUEUV2OSW7Q, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.551183094 (8h5m54.923762439s) [noise=true , address=77.90.31.12:4589 , piece_id=AMXTBSD6WO4UEXIIXO6L5ZQDD3AYUZ75PIMNMX6CXQTPAUS6MKNQ, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.551205177 (8h5m54.923784522s) [noise=true , address=104.36.82.14:4222 , piece_id=S2E5GVHNN4AVBWUCQ7ZF4VTBMCE44VKD3ONII57T52EQAGAQBEBQ, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:48.551419828 (8h5m54.923999173s) [noise=true , address=45.91.116.10:4083 , piece_id=4AIGZG7ABSINNQYFQLPC3ZKRGRD7LUZ6AP4ARHVFHOCYPJY3NPNA, seg_pos={0 0}, upload=6403]: Uploading piece...
Jul 19 06:56:50.313191049 (8h5m56.685770404s) [upload=6403]: Next returned nil segment
Jul 19 06:56:50.313210367 (8h5m56.685789712s) [upload=6403]: Waiting for error group managing segments...
Jul 19 06:56:50.602094226 (8h5m56.974673571s) [noise=true , address=45.91.116.4:4938 , piece_id=ZIJIIBKIGQY2BRI3DVAEWWLPPPUDB2RYMYV2EAD3EZLO2K2P2WTA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.606309361 (8h5m56.978888716s) [noise=true , address=104.36.82.14:4222 , piece_id=S2E5GVHNN4AVBWUCQ7ZF4VTBMCE44VKD3ONII57T52EQAGAQBEBQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.610631160 (8h5m56.983210515s) [noise=true , address=45.91.116.5:4299 , piece_id=OHEAHX6PK3JW6K3BYMJQYK4HDNP6K7DEJOW7EOQKK2TC4P6NW4CA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.613288184 (8h5m56.985867539s) [noise=true , address=45.91.116.8:4045 , piece_id=G5QHS6Y4T2JY7PYOXKUGSZM75F4NWF37L45H434N5WOZKEDUMPWA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.624484997 (8h5m56.997064352s) [noise=true , address=45.91.116.11:4117 , piece_id=BZQFV37K7LE4Z2K54HPA6PBIVNZEC6SBX4HYUE6YGRCHLYUOOEBA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.626515311 (8h5m56.999094656s) [noise=true , address=77.90.31.21:4877 , piece_id=EDLQFKGHOTHTLWL7PPUOMRTA6JIL7TF7HU22JNO2EVVXETFLUHYQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.627277630 (8h5m56.999856975s) [noise=true , address=77.90.31.19:4841 , piece_id=WJMRBXWLIAPBM4YW6SZDQM3A536PDDRMOJF5R4J2LVRDVJPULP4A, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.642423930 (8h5m57.015003285s) [noise=true , address=77.90.31.15:4698 , piece_id=6PREKXY6754B6THCWIZRYR3AO3S2XGPR4GSI7WQ2UNJW2NDWAYEQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.644061784 (8h5m57.016641129s) [noise=true , address=77.90.31.12:4589 , piece_id=AMXTBSD6WO4UEXIIXO6L5ZQDD3AYUZ75PIMNMX6CXQTPAUS6MKNQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.647067965 (8h5m57.01964732s) [noise=true , address=77.90.31.10:4520 , piece_id=PV2J4IO3YYWYVEKXTS4C2CT3BIFNDJNJOYCYDQC5LQC2B4TZCU3A, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.678272179 (8h5m57.050851525s) [noise=true , address=104.36.82.13:4185 , piece_id=JHS6G7M7N5IVY2VPSPHJPVRZQHQJIYBU54KKUW47COZP2YGQ2XGA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.684092075 (8h5m57.05667143s) [noise=true , address=45.91.116.13:4153 , piece_id=MPAR7SWMVQ236Z76HU7FC3LMQ4NRUD3TN64HT3MQRWTWGBQ6YDSQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.701086190 (8h5m57.073665535s) [noise=true , address=104.36.82.15:4259 , piece_id=HAQGLMVTGJXXK334AOVADV45LX4MBWBSDOUHZ7HXYB4CZJ6SWM5A, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil>
Jul 19 06:56:50.701112961 (8h5m57.073692306s) [seg_pos={0 0}, upload=6403]: Segment reached optimal threshold of 13 pieces.
Jul 19 06:56:50.701241187 (8h5m57.073820542s) [noise=true , address=104.36.82.2:4731 , piece_id=MIOJJFTFCRXMSFE4FU2AKBFTPVIX5JEGOJV5FIX5OGUEUV2OSW7Q, seg_pos={0 0}, upload=6403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 19 06:56:50.701311190 (8h5m57.073890545s) [noise=true , address=45.91.116.10:4083 , piece_id=4AIGZG7ABSINNQYFQLPC3ZKRGRD7LUZ6AP4ARHVFHOCYPJY3NPNA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Jul 19 06:56:50.701331519 (8h5m57.073910864s) [noise=true , address=45.91.116.12:4227 , piece_id=A25PGAY3W24PI2WFW6NYMWVSW3NQK4DC5ZJMC4VISWYQICT3SJQA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF
Jul 19 06:56:50.701382838 (8h5m57.073962183s) [seg_pos={0 0}, upload=6403]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 19 06:56:50.701484933 (8h5m57.074064288s) [upload=6403]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 19 06:56:50.701489862 (8h5m57.074069218s) [upload=6403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 19 06:56:51.003338357 (8h5m57.375917732s) [upload=6403]: Done uploading segments...
Jul 19 06:56:51.003368886 (8h5m57.375948241s) [upload=6403]: Upload finished. err: <nil>
Duration: 30.002175831s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.775143314s
FAIL (2025-07-19 06:35:31 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.775167181s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
5.602135556s
FAIL (2025-07-19 06:34:57 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.602163771s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
5.786838173s
FAIL (2025-07-19 06:34:41 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.786931663s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
5.665834073s
FAIL (2025-07-19 06:33:43 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.665856787s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
5.962633488s
FAIL (2025-07-19 06:32:42 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.962661912s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
5.81468406s
FAIL (2025-07-19 06:32:15 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.814712965s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
5.839657364s
FAIL (2025-07-19 06:31:22 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.839681068s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
5.712321462s
FAIL (2025-07-19 06:30:17 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.712348323s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
5.837804805s
FAIL (2025-07-19 06:29:52 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.837830375s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
5.516701306s
FAIL (2025-07-19 06:28:41 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.516724952s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.777333213s
FAIL (2025-07-19 06:27:56 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.777356948s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
5.737007519s
FAIL (2025-07-19 06:27:36 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.737026865s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
5.860366011s
FAIL (2025-07-19 06:26:30 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.860393142s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
5.857545003s
FAIL (2025-07-19 06:25:29 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.857572356s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
5.770461945s
FAIL (2025-07-19 06:24:29 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.770504186s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.928958651s
FAIL (2025-07-19 06:23:52 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.928985292s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.849636226s
FAIL (2025-07-19 06:23:14 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.849662837s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.825194618s
FAIL (2025-07-19 06:22:05 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.825215748s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
5.803701414s
FAIL (2025-07-19 06:21:40 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.803733938s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.808358854s
FAIL (2025-07-19 06:20:38 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.808385084s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.571556217s
FAIL (2025-07-19 06:20:13 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.571580543s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
5.715831542s
FAIL (2025-07-19 06:19:48 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.71585726s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.828311473s
FAIL (2025-07-19 06:19:14 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.828333393s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
5.988409712s
FAIL (2025-07-19 06:18:06 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.988430451s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.727917995s
FAIL (2025-07-19 06:17:01 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.728059496s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.824955745s
FAIL (2025-07-19 06:15:59 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.824981313s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.777356107s
FAIL (2025-07-19 06:15:04 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.777384512s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.806264338s
FAIL (2025-07-19 06:14:47 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.806288594s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.581852962s
FAIL (2025-07-19 06:13:51 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.581890333s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
5.841620416s
FAIL (2025-07-19 06:12:51 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.841644001s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
6.02544202s
FAIL (2025-07-19 06:12:14 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 6.02546885s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.553653844s
FAIL (2025-07-19 06:11:42 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.553673271s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.97256915s
FAIL (2025-07-19 06:11:19 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.972604488s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.837543643s
FAIL (2025-07-19 06:10:56 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.837566528s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
5.839166662s
FAIL (2025-07-19 06:10:14 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.839194887s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
5.856747996s
FAIL (2025-07-19 06:08:59 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.856780399s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
5.771952601s
FAIL (2025-07-19 06:08:29 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.771976376s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
5.767925146s
FAIL (2025-07-19 06:07:56 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.76794834s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
5.756163644s
FAIL (2025-07-19 06:06:56 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.756190674s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
5.744079121s
FAIL (2025-07-19 06:06:23 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.74411061s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.543504315s
FAIL (2025-07-19 06:06:01 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.543528653s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.844815703s
FAIL (2025-07-19 06:05:26 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.844839268s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.908782972s
FAIL (2025-07-19 06:05:25 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
gment. successful: 13 optimal: 13 errs: <nil>
Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments...
Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil>
Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload
Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload
Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments...
Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false
Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment...
Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment.
Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment
Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments...
Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF
Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection
Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF
Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection
Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection
Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments...
Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 5.90880771s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.870156709s
FAIL (2025-07-19 06:04:57 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.870177289s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.834962639s
FAIL (2025-07-19 06:04:30 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.834985984s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
5.759358766s
FAIL (2025-07-19 06:04:16 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
gment. successful: 13 optimal: 13 errs: <nil>
Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments...
Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil>
Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload
Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload
Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments...
Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false
Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment...
Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment.
Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment
Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments...
Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF
Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection
Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF
Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection
Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection
Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments...
Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 5.759380498s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.720467274s
FAIL (2025-07-19 06:03:28 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.720497572s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [X.X.........................................................] duration:
5.836074156s
FAIL (2025-07-19 06:03:12 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
gment. successful: 13 optimal: 13 errs: <nil>
Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments...
Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil>
Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload
Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload
Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments...
Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false
Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment...
Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment.
Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece...
Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment
Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments...
Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil>
Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF
Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection
Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF
Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection
Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection
Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments...
Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 5.836103553s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.924553476s
FAIL (2025-07-19 06:02:59 UTC)
orbiter warning: uplink: metaclient: context deadline exceeded
Duration: 5.924587221s
core
uplink
us1
|