Patch to add a swresample_plugin

This forum is about all development around libVLC.
oviano
Cone that earned his stripes
Cone that earned his stripes
Posts: 120
Joined: 12 Jan 2012 11:12

Patch to add a swresample_plugin

Postby oviano » 04 Nov 2016 09:08

I wasn't happy with libVLC using the "ugly" audio resampler when it had to deal with integer resampling due to the samplerate plugin only supporting floating point.

So in my build of libVLC I am instead using a swresample_plugin based on FFmpeg's libswresample and adapted from this code from a few years ago https://patches.videolan.org/patch/255/ which never seemed to make it into VLC.

Anyway, in case it is useful for anyone, my patch is below. I'm building my libVLC against FFmpeg's libraries rather than libAVs's so be aware that my patch also makes this switch too.

It doesn't unlock anything other than the default libswresample settings but it works nicely for me.

Code: Select all

From 6302fb28813151220e4e8b4057fd42a4bf70777a Mon Sep 17 00:00:00 2001 From: Oliver Collyer <ovcollyer@mac.com> Date: Thu, 3 Nov 2016 20:20:15 +0300 Subject: [PATCH 1/1] add audio filter based on libswresample --- configure.ac | 28 ++++ contrib/src/ffmpeg/rules.mak | 5 +- modules/MODULES_LIST | 1 + modules/audio_filter/Makefile.am | 6 + modules/audio_filter/resampler/swresample.c | 220 ++++++++++++++++++++++++++++ 5 files changed, 257 insertions(+), 3 deletions(-) create mode 100644 modules/audio_filter/resampler/swresample.c diff --git a/configure.ac b/configure.ac index a316e00..0f60869 100644 --- a/configure.ac +++ b/configure.ac @@ -2580,6 +2580,34 @@ then fi dnl +dnl swresample sound resampling and conversion plugin +dnl + +AC_ARG_ENABLE(swresample, + AS_HELP_STRING([--enable-swresample],[libswresample audio resampling and conversion + (default enabled)])) +if test "${enable_swresample}" != "no" +then + PKG_CHECK_MODULES(SWRESAMPLE,[libswresample], + [ + VLC_SAVE_FLAGS + CPPFLAGS="${CPPFLAGS} ${SWRESAMPLE_CFLAGS}" + CFLAGS="${CFLAGS} ${SWRESAMPLE_CFLAGS}" + AC_CHECK_HEADERS(libswresample/swresample.h) + AC_CHECK_HEADERS(libavutil/avutil.h) + VLC_ADD_PLUGIN([swresample]) + VLC_ADD_LIBS([swresample],[$SWRESAMPLE_LIBS]) + VLC_ADD_CFLAGS([swresample],[$SWRESAMPLE_CFLAGS]) + AS_IF([test -n "${ac_cv_ld_bsymbolic}"], [ + VLC_ADD_LIBS([swresample],[${ac_cv_ld_bsymbolic}]) + ]) + VLC_RESTORE_FLAGS + ],[ + AC_MSG_WARN([${SWRESAMPLE_PKG_ERRORS}. swresample not found.]) + ]) +fi + +dnl dnl postproc plugin dnl diff --git a/contrib/src/ffmpeg/rules.mak b/contrib/src/ffmpeg/rules.mak index a7f2fa2..68bdd71 100644 --- a/contrib/src/ffmpeg/rules.mak +++ b/contrib/src/ffmpeg/rules.mak @@ -2,7 +2,7 @@ #Uncomment the one you want #USE_LIBAV ?= 1 -#USE_FFMPEG ?= 1 +USE_FFMPEG ?= 1 ifdef USE_FFMPEG FFMPEG_HASH=HEAD @@ -34,7 +34,6 @@ FFMPEGCONF = \ ifdef USE_FFMPEG FFMPEGCONF += \ - --disable-swresample \ --disable-iconv ifdef HAVE_DARWIN_OS FFMPEGCONF += \ @@ -204,7 +203,7 @@ endif # Build PKGS += ffmpeg -ifeq ($(call need_pkg,"libavcodec >= 55.0.0 libavformat >= 53.21.0 libswscale"),) +ifeq ($(call need_pkg,"libavcodec >= 55.0.0 libavformat >= 53.21.0 libswscale libswresample"),) PKGS_FOUND += ffmpeg endif diff --git a/modules/MODULES_LIST b/modules/MODULES_LIST index e7685a3..19b215c 100644 --- a/modules/MODULES_LIST +++ b/modules/MODULES_LIST @@ -394,6 +394,7 @@ $Id$ * svcdsub: SVCD subtitles decoder * svg: a svg renderer module * svgdec: a svg decoder module + * swresample: Audio resampling filter * swscale: Video scaling filter * syslog: logger output to Syslog * t140: T.140 text encoder diff --git a/modules/audio_filter/Makefile.am b/modules/audio_filter/Makefile.am index 7ee0125..a3d46ba 100644 --- a/modules/audio_filter/Makefile.am +++ b/modules/audio_filter/Makefile.am @@ -118,3 +118,9 @@ libspeex_resampler_plugin_la_LIBADD = $(SPEEXDSP_LIBS) if HAVE_SPEEXDSP audio_filter_LTLIBRARIES += libspeex_resampler_plugin.la endif + +libswresample_plugin_la_SOURCES = audio_filter/resampler/swresample.c +libswresample_plugin_la_CFLAGS = $(AM_CFLAGS) $(SWRESAMPLE_CFLAGS) +libswresample_plugin_la_LIBADD = $(SWRESAMPLE_LIBS) $(LIBM) +libswresample_plugin_la_LDFLAGS = $(AM_LDFLAGS) -rpath '$(audio_filterdir)' +audio_filter_LTLIBRARIES += libswresample_plugin.la diff --git a/modules/audio_filter/resampler/swresample.c b/modules/audio_filter/resampler/swresample.c new file mode 100644 index 0000000..bab9c56 --- /dev/null +++ b/modules/audio_filter/resampler/swresample.c @@ -0,0 +1,220 @@ +/***************************************************************************** + * swresample.c : libswresample audio resampling and conversion + ***************************************************************************** + * Copyright © 2016 Oliver Collyer + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software Foundation, + * Inc., 51 Franklin Street, Fifth Floor, Boston MA 02110-1301, USA. + *****************************************************************************/ + +#ifdef HAVE_CONFIG_H +# include "config.h" +#endif + +#include <vlc_common.h> +#include <vlc_plugin.h> +#include <vlc_aout.h> +#include <vlc_filter.h> + +#include <libavutil/opt.h> +#include <libswresample/swresample.h> + +static int Open (vlc_object_t *); +static int OpenResampler (vlc_object_t *); +static void Close (vlc_object_t *); + +struct swr_sys_t +{ + int16_t in_channel_map; + unsigned int in_sample_rate; + vlc_fourcc_t in_sample_format; + + SwrContext *swr; +}; + +vlc_module_begin () + set_shortname (N_("swresample resampler")) + set_description (N_("swresample resampler")) + set_category (CAT_AUDIO) + set_subcategory (SUBCAT_AUDIO_RESAMPLER) + set_capability ("audio converter", 10) + set_callbacks (Open, Close) + add_submodule () + set_capability ("audio resampler", 10) + set_callbacks (OpenResampler, Close) +vlc_module_end () + +static block_t *Resample (filter_t *, block_t *); + +static enum AVSampleFormat GetAVSampleFormat (vlc_fourcc_t codec) +{ + switch(codec) { + case VLC_CODEC_U8 : return AV_SAMPLE_FMT_U8; + case VLC_CODEC_S32N: return AV_SAMPLE_FMT_S32; + case VLC_CODEC_FL32: return AV_SAMPLE_FMT_FLT; + case VLC_CODEC_FL64: return AV_SAMPLE_FMT_DBL; + case VLC_CODEC_S16N: return AV_SAMPLE_FMT_S16; + } + return AV_SAMPLE_FMT_NONE; +} + +static int InitSwrContext(filter_t *filter) +{ + struct swr_sys_t *sys = (struct swr_sys_t *)filter->p_sys; + SwrContext *s = sys->swr; + + enum AVSampleFormat in_sample_format = + GetAVSampleFormat (filter->fmt_in.audio.i_format); + if (in_sample_format == AV_SAMPLE_FMT_NONE) + { + msg_Err (filter, "unsupported input format"); + return VLC_EGENERIC; + } + + enum AVSampleFormat out_sample_format = + GetAVSampleFormat (filter->fmt_out.audio.i_format); + if (out_sample_format == AV_SAMPLE_FMT_NONE) + { + msg_Err (filter, "unsupported output format"); + return VLC_EGENERIC; + } + + int64_t in_channel_layout = + av_get_default_channel_layout ( + aout_FormatNbChannels (&filter->fmt_in.audio)); + int64_t out_channel_layout = + av_get_default_channel_layout ( + aout_FormatNbChannels (&filter->fmt_out.audio)); + + av_opt_set_int (s, "in_channel_layout", in_channel_layout, 0); + av_opt_set_int (s, "out_channel_layout", out_channel_layout, 0); + av_opt_set_int (s, "in_sample_rate", filter->fmt_in.audio.i_rate, 0); + av_opt_set_int (s, "out_sample_rate", filter->fmt_out.audio.i_rate, 0); + av_opt_set_int (s, "in_sample_fmt", in_sample_format, 0); + av_opt_set_int (s, "out_sample_fmt", out_sample_format, 0); + + int ret; + if ((ret = swr_init (s))) + { + char errbuf[128]; + av_strerror (ret, errbuf, sizeof(errbuf)); + msg_Err (filter, "cannot initialise resampler: %s", errbuf); + return VLC_EGENERIC; + } + + sys->in_sample_format = filter->fmt_in.audio.i_format; + sys->in_sample_rate = filter->fmt_in.audio.i_rate; + sys->in_channel_map = filter->fmt_in.audio.i_physical_channels; + + return VLC_SUCCESS; +} + +static int Open (vlc_object_t *obj) +{ + filter_t * filter = (filter_t *)obj; + + if (filter->fmt_in.audio.i_rate == filter->fmt_out.audio.i_rate) + return VLC_EGENERIC; + return OpenResampler (obj); +} + +static int OpenResampler (vlc_object_t *obj) +{ + filter_t * filter = (filter_t *)obj; + + struct swr_sys_t *s = malloc (sizeof (struct swr_sys_t)); + if (unlikely(s == NULL)) + { + msg_Err (obj, "cannot malloc resampler context"); + return VLC_EGENERIC; + } + + s->swr = swr_alloc(); + if (unlikely (s->swr == NULL)) + { + msg_Err (obj, "cannot create resampler context"); + free (s); + return VLC_EGENERIC; + } + + filter->p_sys = (filter_sys_t *)s; + int ret = InitSwrContext(filter); + if (unlikely (ret != VLC_SUCCESS)) + { + Close(obj); + return VLC_EGENERIC; + } + + filter->pf_audio_filter = Resample; + return VLC_SUCCESS; +} + +static void Close (vlc_object_t *obj) +{ + filter_t *filter = (filter_t *)obj; + struct swr_sys_t *sys = (struct swr_sys_t *) filter->p_sys; + SwrContext *s = sys->swr; + + swr_close (s); + swr_free (&s); + + free(sys); + filter->p_sys = NULL; +} + +static block_t *Resample (filter_t *filter, block_t *in) +{ + block_t *out = NULL; + struct swr_sys_t *sys = (struct swr_sys_t *) filter->p_sys; + SwrContext *s = sys->swr; + + if (unlikely ( + sys->in_sample_rate != filter->fmt_in.audio.i_rate || + sys->in_sample_format != filter->fmt_in.audio.i_format || + sys->in_channel_map != filter->fmt_in.audio.i_physical_channels)) + { + swr_close (s); + + int ret = InitSwrContext (filter); + if (unlikely (ret != VLC_SUCCESS)) + goto error; + } + + const size_t framesize = filter->fmt_out.audio.i_bytes_per_frame; + int out_nb_samples = swr_get_out_samples(s, in->i_nb_samples); + out = block_Alloc (out_nb_samples * framesize); + if (unlikely(out == NULL)) + goto error; + + int ret = swr_convert (s, &out->p_buffer, out_nb_samples, + (const uint8_t**)&in->p_buffer, in->i_nb_samples); + if (ret < 0) + { + char errbuf[128]; + av_strerror (ret, errbuf, sizeof(errbuf)); + msg_Err (filter, "cannot resample: %s", errbuf); + block_Release (out); + out = NULL; + goto error; + } + + out->i_nb_samples = ret; + out->i_buffer = out->i_nb_samples * framesize; + out->i_pts = in->i_pts; + out->i_length = ret * CLOCK_FREQ / filter->fmt_out.audio.i_rate; + +error: + block_Release (in); + return out; +} -- 2.7.4

Jean-Baptiste Kempf
Site Administrator
Site Administrator
Posts: 37523
Joined: 22 Jul 2005 15:29
VLC version: 4.0.0-git
Operating System: Linux, Windows, Mac
Location: Cone, France
Contact:

Re: Patch to add a swresample_plugin

Postby Jean-Baptiste Kempf » 07 Dec 2016 00:39

How is the performance with this resampler?
Jean-Baptiste Kempf
http://www.jbkempf.com/ - http://www.jbkempf.com/blog/category/Videolan
VLC media player developer, VideoLAN President and Sites administrator
If you want an answer to your question, just be specific and precise. Don't use Private Messages.

oviano
Cone that earned his stripes
Cone that earned his stripes
Posts: 120
Joined: 12 Jan 2012 11:12

Re: Patch to add a swresample_plugin

Postby oviano » 07 Dec 2016 05:33

All I can say is that it works. I haven't measured anything regarding its performance.

More importantly, I'm happy that a module called "ugly" no longer needs to be near my code or displayed in the logs :)


Return to “Development around libVLC”

Who is online

Users browsing this forum: No registered users and 17 guests