教育App Flutter点播组件实战

2019/7/29 9:38:41 人评论 次浏览 分类:Flutter                 进angular qq群大神亲自指导     angular交流群②

来源 | 腾讯在线教育技术

教育从去年开始接入Flutter,今年上半年重构腾讯课堂和企鹅辅导iPad端,80%代码都采用Flutter实现,对于教育最重要的点播功能同样也需要迁移到Flutter上进行渲染。

目前正在研究的实现渲染的方案主要有2种形式PlatformView和Texture Widget。下面文章就先大概讲述一下Flutter的渲染框架原理和实现,然后会对这两种方案进行对比分析。

Flutter渲染框架和原理

Flutter的框架主要包括Framework和Engine两层,应用是基于Framework层开发的,Framework负责渲染中的Build,Layout,Paint,生成Layer等。Engine层是C++实现的渲染引擎,负责把Framework生成的Layer组合,生成纹理,然后通过OpenGL接口向GPU提交渲染数据。

Flutter:Framework的最底层,提供工具类和方法Painting :封装了Flutter Engine提供的绘制接口,主要是为了在绘制控件等固定样式的图形时提供更直观、更方便的接口Animation :动画相关的类Gesture :提供了手势识别相关的功能,包括触摸事件类定义和多种内置的手势识别器Rendering:渲染库,Flutter的控件树在实际显示时会转换成对应的渲染对象(RenderObject)树来实现布局和绘制操作

渲染原理

当GPU发出Vsync信号时,会执行Dart代码绘制新UI,Dart会被执行为Layer Tree,然后经过Compositor合成后交由Skia引擎渲染处理为GPU数据,最后通过GL/Vulkan发给GPU,具体流程如下:

当需要更新UI的时候,Framework通知Engine,Engine会等到下个Vsync信号到达的时候通知Framework,Framework进行animations,build,layout,compositing,paint,最后生成layer提交给Engine。Engine再把layer进行组合,生成纹理,最后通过OpenGL接口提交数据给GPU,具体流程如下:


接下来分别分析一下两个方案的各自的特点以及使用的方式。

PlatformView

PlatformView是Flutter官方在1.0版本推出的组件,以解决开发者想在Flutter中嵌入Android和iOS平台原生View的Widget。例如想嵌入地图、视频播放器等原生组件,对于想尝试Flutter,但是又想低成本的迁移复杂组件的团队,可以尝试PlatformView,在 Dart 中的类对应到 iOS 和 Android 平台分别是UIKitView和AndroidView。

那么PlatformView在点播功能中应该怎么实现,如下图所示:

其中的ARMPlatformView代表业务View。

Dart层

1.创建关联类

关联类的作用是Native和Dart侧的桥梁,其中id需要和Native获取对应。

  1. class VodPlayerController {


  2. VodPlayerController._(int id)

  3. : _channel = MethodChannel('ARMFlutterVodPlayerView_$id');


  4. final MethodChannel _channel;


  5. Future<void> play(String url) async {

  6. return _channel.invokeMethod('play', url);

  7. }


  8. Future<void> stop() async {

  9. return _channel.invokeMethod('stop');

  10. }

  11. }

2.创建Callback

  1. typedef void VodPlayerViewWidgetCreatedCallback(VodPlayerController controller);

3.创建Widget布局

  1. class VodVideoWidget extends StatefulWidget {

  2. final VodPlayerViewWidgetCreatedCallback callback;

  3. final x;

  4. final y;

  5. final width;

  6. final height;


  7. VodVideoWidget({

  8. Key key,

  9. @required this.callback,

  10. @required this.x,

  11. @required this.y,

  12. @required this.width,

  13. @required this.height,

  14. });


  15. @override

  16. _VodVideoWidgetState createState() => _VodVideoWidgetState();

  17. }


  18. class _VodVideoWidgetState extends State<VodVideoWidget> {


  19. @override

  20. Widget build(BuildContext context) {

  21. return UiKitView(

  22. viewType: 'ARMFlutterVodPlayerView',

  23. onPlatformViewCreated: _onPlatformViewCreated,

  24. creationParams: <String,dynamic>{

  25. 'x': widget.x,

  26. 'y': widget.y,

  27. 'width': widget.width,

  28. 'height': widget.height,

  29. },

  30. creationParamsCodec: new StandardMessageCodec(),

  31. );

  32. }


  33. void _onPlatformViewCreated(int id){

  34. if(widget.callback == null) {

  35. return;

  36. }

  37. widget.callback(VodPlayerController._(id));

  38. }

  39. }

Native层

1.注册ViewFactory

  1. @implementation ARMFlutterVodPlugin


  2. + (void)registerWithRegistrar:(nonnull NSObject<FlutterPluginRegistrar> *)registrar {

  3. ARMFlutterVodPlayerFactory* vodFactory =

  4. [[ARMFlutterVodPlayerFactory alloc] initWithMessenger:registrar.messenger];

  5. [registrar registerViewFactory:vodFactory withId:@"ARMFlutterVodPlayerView"];

  6. }

  7. @end

2.注册Plugin

  1. + (void)registerWithRegistry:(NSObject<FlutterPluginRegistry>*)registry {

  2. [ARMFlutterVodPlugin registerWithRegistrar:[registry registrarForPlugin:@"ARMFlutterVodPlugin"]];

  3. }

3.ViewFactory实现

  1. @implementation ARMFlutterVodPlayerFactory


  2. - (instancetype)initWithMessenger:(NSObject<FlutterBinaryMessenger> *)messager {

  3. self = [super init];

  4. if (self) {

  5. _messenger = messager;

  6. }

  7. return self;

  8. }


  9. - (NSObject<FlutterMessageCodec> *)createArgsCodec {

  10. return [FlutterStandardMessageCodec sharedInstance];

  11. }


  12. - (nonnull NSObject<FlutterPlatformView> *)createWithFrame:(CGRect)frame viewIdentifier:(int64_t)viewId arguments:(id _Nullable)args {

  13. return [[ARMFlutterVodPlayerView alloc] initWithWithFrame:frame viewIdentifier:viewId arguments:args binaryMessenger:self.messenger];

  14. }


  15. @end

4.View实现

  1. @implementation ARMFlutterVodPlayerView


  2. - (instancetype)initWithWithFrame:(CGRect)frame viewIdentifier:(int64_t)viewId arguments:(id)args binaryMessenger:(NSObject<FlutterBinaryMessenger> *)messenger{

  3. if (self = [super init]) {

  4. NSDictionary *dic = args;

  5. CGFloat x = [dic[@"x"] floatValue];

  6. CGFloat y = [dic[@"y"] floatValue];

  7. CGFloat width = [dic[@"width"] floatValue];

  8. CGFloat height = [dic[@"height"] floatValue];

  9. ARMFlutterVodManager.shareInstance.mainView.frame = CGRectMake(x, y, width, height);

  10. NSString* channelName = [NSString stringWithFormat:@"ARMFlutterVodPlayerView_%lld", viewId];

  11. _channel = [FlutterMethodChannel methodChannelWithName:channelName binaryMessenger:messenger];

  12. __weak __typeof__(self) weakSelf = self;

  13. [_channel setMethodCallHandler:^(FlutterMethodCall * call, FlutterResult result) {

  14. [weakSelf onMethodCall:call result:result];

  15. }];

  16. }


  17. return self;

  18. }



  19. - (nonnull UIView *)view {

  20. return ARMFlutterVodManager.shareInstance.mainView;

  21. }


  22. - (void)onMethodCall:(FlutterMethodCall*)call result:(FlutterResult)result{

  23. if ([[call method] isEqualToString:@"play"]) {

  24. NSString *url = [call arguments];

  25. [ARMFlutterVodManager.shareInstance play:url];

  26. } else {

  27. result(FlutterMethodNotImplemented);

  28. }

  29. }


  30. @end

Texture Widget

基于纹理实现视频渲染,Flutter官方提供的video_player则是通过这种方式实现的,以iOS为例,Native需要提供一个CVPixelBufferRef给Texture Widget,具体实现流程如下图所示:

其中的ARMTexture是业务提供CVPixelBufferRef,具体实现步骤主要是1.继承FlutterTexture2.管理已注册textures集合

  1. textures = [self.registrar textures];

3.获得textureId

  1. self.textureId = [textures registerTexture:self];

4.重写

  1. - (CVPixelBufferRef _Nullable)copyPixelBuffer

返回CVPixelBufferRef5.通知Texture获取CVPixelBufferRef

  1. - (void)onDisplayLink {

  2. [textures textureFrameAvailable:self.textureId];

  3. }

Dart层

  1. MethodChannel _globalChannel = MethodChannel("ARMFlutterTextureVodPlayer");


  2. class _ARMPlugin {

  3. MethodChannel get channel => MethodChannel("ARMFlutterTextureVodPlayer/$textureId");


  4. int textureId;


  5. _ARMPlugin(this.textureId);


  6. Future<void> play() async {

  7. await channel.invokeMethod("play");

  8. }


  9. Future<void> pause() async {

  10. await channel.invokeMethod("pause");

  11. }


  12. Future<void> stop() async {

  13. await channel.invokeMethod("stop");

  14. }


  15. Future<void> setNetworkDataSource(

  16. {String uri, Map<String, String> headers = const {}}) async {

  17. await channel.invokeMethod("setNetworkDataSource", <String, dynamic>{

  18. "uri": uri,

  19. "headers": headers,

  20. });

  21. }

  22. }

Native层

1.注册Plugin

  1. @implementation ARMFlutterTextureVodPlugin


  2. - (instancetype)initWithRegistrar:(NSObject <FlutterPluginRegistrar> *)registrar {

  3. self = [super init];

  4. if (self) {

  5. self.registrar = registrar;

  6. }


  7. return self;

  8. }


  9. + (instancetype)pluginWithRegistrar:(NSObject <FlutterPluginRegistrar> *)registrar {

  10. return [[self alloc] initWithRegistrar:registrar];

  11. }


  12. + (void)registerWithRegistrar:(NSObject <FlutterPluginRegistrar> *)registrar {

  13. FlutterMethodChannel *channel = [FlutterMethodChannel

  14. methodChannelWithName:@"ARMFlutterTextureVodPlayer"

  15. binaryMessenger:[registrar messenger]];

  16. ARMFlutterTextureVodPlugin *instance = [ARMFlutterTextureVodPlugin pluginWithRegistrar:registrar];

  17. [registrar addMethodCallDelegate:instance channel:channel];

  18. }


  19. - (void)handleMethodCall:(FlutterMethodCall *)call result:(FlutterResult)result {


  20. }


  21. @end

2.管理Texture/获取TextureId

  1. + (instancetype)armWithRegistrar:(NSObject <FlutterPluginRegistrar> *)registrar {

  2. return [[self alloc] initWithRegistrar:registrar];

  3. }


  4. - (instancetype)initWithRegistrar:(NSObject <FlutterPluginRegistrar> *)registrar {

  5. if (self = [super init]) {

  6. self.registrar = registrar;

  7. textures = [self.registrar textures];

  8. self.textureId = [textures registerTexture:self];

  9. NSString *channelName = [NSString stringWithFormat:@"ARMFlutterTextureVodPlayer/%lli", self.textureId];

  10. channel = [FlutterMethodChannel methodChannelWithName:channelName binaryMessenger:[registrar messenger]];

  11. __weak typeof(&*self) weakSelf = self;

  12. [channel setMethodCallHandler:^(FlutterMethodCall *call, FlutterResult result) {

  13. [weakSelf handleMethodCall:call result:result];

  14. }];

  15. }

  16. return self;

  17. }

3.重写copyPixelBuffer

  1. - (CVPixelBufferRef _Nullable)copyPixelBuffer {

  2. CVPixelBufferRef newBuffer = [self.vodPlayer framePixelbuffer];

  3. if (newBuffer) {

  4. CFRetain(newBuffer);

  5. CVPixelBufferRef pixelBuffer = latestPixelBuffer;

  6. while (!OSAtomicCompareAndSwapPtrBarrier(pixelBuffer, newBuffer, (void **) &latestPixelBuffer)) {

  7. pixelBuffer = latestPixelBuffer;

  8. }


  9. return pixelBuffer;

  10. }

  11. return NULL;

  12. }

4.调用textureFrameAvailable

这里是需要主动调用的,告诉TextureRegistry更新画面。

  1. displayLink = [CADisplayLink displayLinkWithTarget:self selector:@selector(onDisplayLink)];

  2. displayLink.frameInterval = 1;

  3. [displayLink addToRunLoop:[NSRunLoop currentRunLoop] forMode:NSRunLoopCommonModes];

  1. - (void)onDisplayLink {

  2. [textures textureFrameAvailable:self.textureId];

  3. }

性能对比

播放同一段MP4视频PlatformView和Texture Widget性能对比,Texture Widget性能相对差一些,分析主要原因是因为CVPixelBufferRef提供给Flutter的Texture,Native到Flutter会经过GPU->CPU->GPU的拷贝过程,1.0版本数据对比如下:

遇到的问题

Flutter播放器接入到课堂iPad中采用的是Texure的方案,在实现PlatformView和TextureWidget两个方案的时候,主要遇到了以下几个问题。

PlatformView内存增长问题

课堂在连续播放视频之后,出现内存暴增问题,主要原因是OpenGL操作都需要设置[EAGLContext setCurrentContext:context_]在IOSGLRenderTarget析构的时候,没有设置context上下文。

课堂直播场景退出之后,前面Flutter页面出现黑屏

课堂直播课退出之后回到上一个Flutter页面出现页面黑屏,直播视频渲染也是采用OpenGL,当直播退出的时候不仅需要设置context,还需要清空帧缓冲区,重置纹理,销毁代码如下:

  1. EAGLContext *prevContext = [EAGLContext currentContext];

  2. [EAGLContext setCurrentContext:_context];

  3. _renderer = nil;


  4. glBindTexture(GL_TEXTURE_2D, 0);

  5. glBindBuffer(GL_ARRAY_BUFFER, 0);

  6. glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 0);


  7. if (_framebuffer) {

  8. glDeleteFramebuffers(1, &_framebuffer);

  9. _framebuffer = 0;

  10. }


  11. if (_renderbuffer) {

  12. glDeleteRenderbuffers(1, &_renderbuffer);

  13. _renderbuffer = 0;

    }


  14. if (_program) {

  15. glDeleteProgram(_program);

  16. _program = 0;

  17. }


  18. _context = nil;


  19. [EAGLContext setCurrentContext:prevContext];


Texture Widget内存相比PlatformView性能相对差一些

主要原因是因为CVPixelBufferRef提供给Flutter的Texture,Native到Flutter会经过GPU->CPU->GPU的拷贝过程,所以我们将Native生成TextureID->拷贝生成PixelBuffer->生成新的TextureID改为直接通过Native生成TextureID->渲染,减少多次拷贝引起的内存问题,经过优化Texture Widget的整体性能优于PlatformView。2.0优化后数据对比如下:

总结

目前基于教育自研的播放器ARMPlayer的Flutter播放器Plugin已经在腾讯课堂iPad中使用,采用优化后的Texture Widget方案,Texture Widget是官方推荐的方式,不管是视频,还是图片都可以用Texture,方便扩展,同时通过纹理形式贴到LayerTree上保证平台无关,多端可复用,优化后的Texture Widget性能也优于PlatformView,Flutter播放器Plugin是教育客户端中台(大前端和点播)结合的一个新的尝试。

本文链接 http://www.ngui.cc/news/show-15875.html

教程有问题或不明白可以在评论区评论站长会用心修改的

相关资讯

    暂无相关的资讯...