Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 868828 reports (Page 11624 of 17377)

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [................X..............X.....X......X...............] duration: 3.352495133s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:26:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.352518638s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.916588332s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:26:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.916615493s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 16.236643637s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:26:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 16.236687721s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [.................X..............X.....X......X..............] duration: 3.480850066s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:25:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.48087293s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.791572776s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:25:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.791590039s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 11.79007419s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:25:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 11.790105099s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [..................X..............X.....X......X.............] duration: 3.534434374s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:25:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.534464522s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.956448356s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:25:10 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.956480006s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 13.477253436s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:25:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 13.477283172s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.636091559s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:24:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.636122467s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 13.861349031s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:24:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 13.861383709s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [...................X..............X.....X......X............] duration: 3.471990117s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:24:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.472022789s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.694819932s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:24:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.694858937s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [....................X..............X.....X......X...........] duration: 3.438880273s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:24:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.438975453s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 12.398089513s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:24:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 12.398122176s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.855614691s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:23:57 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.855642022s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.874412249s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:23:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.874440463s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 14.005618737s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:23:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 14.005652501s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.177452366s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:23:05 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.177477463s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.640069792s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:22:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.640097655s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [.....................X..............X.....X......X..........] duration: 3.413868441s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:22:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.413921592s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 10.152489993s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:22:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 10.152511454s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.........X.................................................] duration: 6.042323791s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:22:08 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.042437838s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [......................X..............X.....X......X.........] duration: 3.534049518s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:22:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.534086519s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.90643203s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:22:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.906462259s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X.........X................................................] duration: 5.822063912s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:21:43 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.822093769s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.277339592s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:21:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.277371013s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X.........X...............................................] duration: 6.134392639s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:21:14 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.134417637s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [.......................X..............X.....X......X........] duration: 3.70431897s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:21:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.704349557s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.890464043s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:20:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.890491356s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.895268277s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:20:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.895429154s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [........................X..............X.....X......X.......] duration: 3.581628442s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:20:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.581787043s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 4.674363384s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:20:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.674400244s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [.........................X..............X.....X......X......] duration: 3.534252578s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:20:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.534279861s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X.........X..............................................] duration: 6.084968727s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:20:01 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.085001018s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X.........X.............................................] duration: 5.808296835s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:19:41 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.808372708s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.827186192s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:19:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.827214656s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.133697982s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:19:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.133733079s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [..........................X..............X.....X......X.....] duration: 3.602473236s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:19:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.602502172s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m1.550406085s trace: https://jaeger.dev.storj.io/trace/2278184cf7cb09d3 FAIL (2025-12-09 09:19:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.957067453 (89h49m23.828150528s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.958192827 (89h49m23.829275862s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.963154113 (89h49m23.834237159s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.964026705 (89h49m23.83510977s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.967189518 (89h49m23.838272563s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969161493 (89h49m23.840244548s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.969242788 (89h49m23.840325834s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.970553794 (89h49m23.841636849s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.972813026 (89h49m23.843896081s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:23.978089925 (89h49m23.84917298s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Uploading piece... Dec 9 09:18:24.785720741 (89h49m24.656803786s) [upload=78358]: Next returned nil segment Dec 9 09:18:24.785736831 (89h49m24.656819876s) [upload=78358]: Waiting for error group managing segments... Dec 9 09:18:25.168467403 (89h49m25.039550448s) [noise=true , address=185.167.204.206:4049 , piece_id=MEZGGAN2ZFVW535D3QU5FIF5X2XNI67T7FMNIJQMMXLHF5NN7KCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185302831 (89h49m25.056385876s) [noise=true , address=185.167.204.206:4054 , piece_id=CBV3DDAZUBWN4RVNVA37BC5ESANQPT4MYGHWMNTO4AZJ32ZZ7LWQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.185793916 (89h49m25.056876961s) [noise=true , address=185.167.204.206:4057 , piece_id=AVPMO43TLI7KCC52Y4OH2W2G3RLMGLBU7BTGPEO5FG2VUN7272DA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.186032270 (89h49m25.057115315s) [noise=true , address=185.167.204.206:4050 , piece_id=LBHTVVRMFZERIJPYLJGIZB6FC4YERLQI4YL2OMUXKN37OLQHDHDA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.188698217 (89h49m25.059781262s) [noise=true , address=185.167.204.205:4016 , piece_id=T4SMB72E6JS76626B3SOUTKCFK6OHRPEQSKOLCBBQW5XY5PO3TNA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.198062969 (89h49m25.069146014s) [noise=true , address=185.167.204.205:4024 , piece_id=Q53WJVADCT7IWGNY7OUXAA4VGI6ARFLTGDLM2KM6LBE2HBBA3DAQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.201189182 (89h49m25.072272227s) [noise=true , address=185.167.204.206:4060 , piece_id=TR5JMA65VL3X2JYKINUGVORZFKKBKV2NY2LBLHPQJS5Z4P2NJHWA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.202370381 (89h49m25.073453426s) [noise=true , address=185.167.204.206:4069 , piece_id=LD44HDOWUF7N2LWP3MU4DSDXO3YAI6KVCUP2US3A6OX6WIGW3XYQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.205089780 (89h49m25.076172816s) [noise=true , address=185.167.204.205:4032 , piece_id=R75QHC7MZRLI6HXHLCBWU5ZCKUQQI35I6URWVG7WKDIAFWIPN7XQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.207830660 (89h49m25.078913705s) [noise=true , address=185.167.204.206:4037 , piece_id=NW4QWZFM7JA6UF7JNUWNPEYL76BLWLB5KF3VLHGJZSCORTKL5SOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.208925113 (89h49m25.080008159s) [noise=true , address=185.167.204.205:4036 , piece_id=JE2RTZWLPINK4DZEIN5VZSF7NQNDIJ4EEEHLNXIGCXZAVVK23RGQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.209472386 (89h49m25.080555431s) [noise=true , address=185.167.204.206:4065 , piece_id=LQIMSCI7CLY57EOLIERM5NSR64QSNR64FD6H2L6OUQV6VANBP77Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.211577764 (89h49m25.082660809s) [noise=true , address=185.167.204.205:4017 , piece_id=MILTVCWHWQJBMZFZPFXXCDSTPWRYHDHBFQEOPO6YG5PQTMD3RONQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.219514148 (89h49m25.090597183s) [noise=true , address=185.167.204.205:4015 , piece_id=YU77JIVCOQ37G52QR4RKZTQ2WY7ME4JPHBM7RTUHBHAM5A4DCFDQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.769337000 (89h49m25.640420045s) [noise=true , address=81.171.14.70:4039 , piece_id=QFA6JIAMTETMRGCCK6PJDUKWH43PZJOQQ4X5Y5ILCGL5R37OYTUQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.772473423 (89h49m25.643556468s) [noise=true , address=81.171.14.70:4032 , piece_id=FJAX2O76ZV2L5FWWXABF7FGT5JUQ6A3YDMKJUQSREBEJPIGQP7RA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800364573 (89h49m25.671447608s) [noise=true , address=81.171.14.70:4030 , piece_id=6VDC74RCXTDZDGRMAGDR6HKRMEZAUCVM2KXIORPRFHOUPZ4SRURQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.800677198 (89h49m25.671760243s) [noise=true , address=81.171.14.70:4047 , piece_id=WZJLBCHCHZHOHL5IC66ZWGIF4EOHNOEH3XBVBXIZCV65YRSTNPBA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.810545961 (89h49m25.681629006s) [noise=true , address=81.171.14.70:4046 , piece_id=CZRVAU2PNZBUSVCYX5MOV72SZXID4GPJ7BDPPD4NIJ235EY62YMA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:25.838216351 (89h49m25.709299396s) [noise=true , address=81.171.14.70:4031 , piece_id=TL65276T2XDWZUTNUZ7IBI5D4BSAMB5S6W6G5CT3NZ6JF2WSESOA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.250350354 (89h49m26.121433399s) [noise=true , address=81.171.14.70:4036 , piece_id=3PRKMFHXBCM52IQ3IUGSQR5G727WYKUTVJBVC5QERPZ2WFCGSA3Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.277784646 (89h49m26.148867691s) [noise=true , address=81.171.14.70:4048 , piece_id=PTOL6CIPLQS6LTCCLI6PFEXOAOXHTMUQ4SYTRRJRWSXTED4VHMFQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.519380506 (89h49m26.390463552s) [noise=true , address=89.149.201.147:4005 , piece_id=HVHGMPYGLWAMDJVZTNUVXZ3GCPWDKDF3SBCJO2ILKKQGSPAYEBFA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.910352793 (89h49m26.781435828s) [noise=true , address=89.149.201.147:4001 , piece_id=PJ3CG6DO2HMSDREM23CEEBEDEEMREOBD2C2WMZSM2B5MSRASY6GQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.962433886 (89h49m26.833516921s) [noise=true , address=89.149.201.147:4004 , piece_id=JNEKFVLJN5FTXT46CVFQVGOA5BTDNL6TDB655V6WWMZWT5LC7FBQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974479733 (89h49m26.845562768s) [noise=true , address=89.149.201.147:4023 , piece_id=LQPBNSGESUY2CONNWGJWPE67S76QATD3T2DICJPYJ3H4J2NRCWKQ, seg_pos={0 0}, upload=78358]: Done uploading piece. err: <nil> Dec 9 09:18:26.974507296 (89h49m26.845590341s) [seg_pos={0 0}, upload=78358]: Segment reached optimal threshold of 26 pieces. Dec 9 09:18:26.974751972 (89h49m26.845835017s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974762383 (89h49m26.845845438s) [noise=true , address=89.149.201.147:4007 , piece_id=PY2KA7ZSI422BIOOTFWMLFY3LVQIFS4DC2VWTBR5HUN5C36DR53Q, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974851171 (89h49m26.845934226s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 9 09:18:26.974873604 (89h49m26.845956669s) [noise=true , address=89.149.201.147:4019 , piece_id=JI735NFBOCR6YSMRDKNIH653LHV3D5LGY34PJMPISBZE7TV57MCA, seg_pos={0 0}, upload=78358]: Long tail cancellation due to optimal threshold Dec 9 09:18:26.974923419 (89h49m26.846006464s) [seg_pos={0 0}, upload=78358]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 09:18:26.975251043 (89h49m26.846334138s) [upload=78358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 09:18:26.975262835 (89h49m26.84634587s) [upload=78358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:19:23.276531408 (89h50m23.147614453s) [upload=78358]: Done uploading segments... Dec 9 09:19:23.276661235 (89h50m23.14774428s) [upload=78358]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m1.550432514s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.836106711s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:19:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.836132983s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X.........X............................................] duration: 6.160466359s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:18:35 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.1604985s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [...........................X..............X.....X......X....] duration: 3.447149935s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:18:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.447178007s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.75960999s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:18:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.759638727s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [............................X..............X.....X......X...] duration: 3.512482472s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:17:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.512512578s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X.........X...........................................] duration: 5.930415183s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:17:35 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.930442586s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [.............................X..............X.....X......X..] duration: 3.714696693s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:17:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.714724295s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X.........X..........................................] duration: 5.680660935s trace: https://jaeger.dev.storj.io/trace/3a6fda9f00eec135 FAIL (2025-12-09 09:17:15 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.680702935s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [..............................X..............X.....X......X.] duration: 3.41281771s trace: https://jaeger.dev.storj.io/trace/62479de42d709d4f FAIL (2025-12-09 09:17:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.412843339s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.060392516s trace: https://jaeger.dev.storj.io/trace/5e8cc43df2e9fadd FAIL (2025-12-09 09:17:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.060423775s core uplink us-select-1 us1-placement-20

1
...
11622
11623
11624
11625
11626
...
17377