hevc

Ramzi Khsib  LinkedIn profile

AWS Elemental website

---------------------------------------------------

Join our LinkedIn Group so that you can get the latest video insider news and participate in the discussion.

Email thevideoinsiders@beamr.com to be a guest on the show.

Learn more about Beamr

Brian Alvarez LinkedIn profile

Vittorio Giovara Blog

---------------------------------------------------

Join our LinkedIn Group so that you can get the latest video insider news and participate in the discussion.

Email thevideoinsiders@beamr.com to be a guest on the show.

Learn more about Beamr

Thierry Fautier LinkedIn profile 

Harmonic website

Ben Mesander LinkedIn profile 

Wowza website

Walid Hamri LinkedIn profile 

SeaChange website

Wade Wan LinkedIn profile 

Broadcom website

Our previous panel on extending the life of H.264 is here

---------------------------------------------------

Join our LinkedIn Group so that you can get the latest video insider news and participate in the discussion.

Email thevideoinsiders@beamr.com to be a guest on the show.

Learn more about Beamr

Gregg Heil LinkedIn profile

Encoding.com, Gregg's company, is here

Beamr CABR on encoding.com is here

---------------------------------------------------

Join our LinkedIn Group so that you can get the latest video insider news and participate in the discussion.

Email thevideoinsiders@beamr.com to be a guest on the show.

Learn more about Beamr

Learn about Unified Patents here

Check out Unified Patents Objective PAtent Landscape OPAL tool

Read the Independent economic study for HEVC royalties

Shawn Ambwani LinkedIn profile

Related episode: VVC, HEVC & other MPEG codec standards

The Video Insiders LinkedIn Group is where thousands of your peers are discussing the latest video technology news and sharing best practices. Click here to join

--------------------------------------

Would you like to be a guest on the show? Email: thevideoinsiders@beamr.com

Learn more about Beamr

--------------------------------------

TRANSCRIPT (edited slightly for readability)

Narrator: 00:00 The Video Insiders is the show that makes sense of all that is happening in the world of online video as seen through the eyes of a second generation codec nerd and a marketing guy who knows what I-frames and macro blocks are. Here are your hosts, Mark Donnigan and Dror Gill.

Mark Donnigan: 00:19 Well, welcome back to The Video Insiders. Dror, how you doing today? I'm doing great. How are you Mark? I am doing awesome. As always. I am super pleased to welcome Shawn Ambwani who is co-founder of Unified Patents and Shawn is gonna tell us all about what Unified Patents does and we're going to dive into, you know, just a really tremendous discussion. But Shawn, Welcome to the podcast!

Shawn Ambwani: 00:46 Hey guys. Thanks Mark. Thanks for, for allowing me to participate on your wonderful podcast. I look at this as similar to 'All Things Considered' and 'How I built this', two of my favorite podcasts.

Mark Donnigan: 01:01 Those are awesome podcasts by the way. What an honor? Yeah. Wow. The level that I expect you guys to be at in traffic very shortly. That's right. Well, we hope so to. Well, why don't you introduce yourself you know, and give us a quick snapshot of your background and then let's let's hear about what Unified Patents is doing.

Shawn Ambwani: 01:23 It's kind of a, I have an interesting I mean some might say not so interesting, but I think it's interesting background related to this area since, you know, the first startup that I did and the second one were all related to MPEG4. So I co-founded a company called Envivio, which way back when was actually one of the original MPEG4 companies when they just had simple profile actually out there doing encoders and decoders. And then I went to a Korean company called NexStreaming, which actually still exists, which is doing encoders as well, but more for the mobile space and decoders. So it's an area I'm quite familiar with. I wasn't really being an attorney back then. Now I'm kind of more of an attorney than I was back then, but I tried to avoid being an attorney as much as possible in general.

Shawn Ambwani: 02:16 And basically I helped co-found a company called Unified Patents. And what unified patents does is it gets contributions from member companies as well as it allows small companies to join for free and they participate in joining what we call zones. And these different zones are intended to protect against what we consider unsubstantiated or invalid patent assertions. And the goal of these zones is to deter those from occurring in the first place. So if you imagine the kind of a technology area, let it be content or let it be video codec in this case or other things as having a bunch of companies that have a common interest in maintaining, you know, patents and ensuring patents that are asserted in that space are valid, which means that no one invented the idea beforehand. And also that it's fairly priced and you know, people are explaining the rationale behind what they're doing and they're not basically just attempting to get people to settle out, not because the assertions are valid or good, but simply because the cost of litigation is so high when it comes to patents.

Shawn Ambwani: 03:35 And we want to deter that type of activity because there's been a lot of investment in that activity so far. In fact, most litigation's are by NPE's and so Unified started by doing those zones and, and, and we've have a bunch of them now. We just launched an open source zone in fact, but with you know, Linux foundation and OIN and IBM and others and the video codec zone was something that we were thinking about for a long time. It's something that I'm very familiar with from my past dealings with MPEG LA and other pools. And it was a big issue I think. And it has been a constant issue, which is how do you deal with multiple pools or multiple people asking for money in a standard? How do you deal with the pricing of it? Especially if you're smaller entities and you don't have the information that may be larger companies might have.

Shawn Ambwani: 04:26 How do you deal with that and how do you deal with all the invalid assertions that are being done or declarations that occur in this area? How do you figure out who you have to pay? And how much you have to pay. All these add a level of complexity to deploying these standards, which makes adoption harder and creates the uncertainty that causes people to go to proprietary solutions, which I think is a negative in the end. So that's why Unified Patents really created this area and created the video codec zone. And basically we've been pretty, I think, successful so far now actually going through and doing each one of the things we said we were going to do.

Dror Gill: 05:06 So what, what are those things basically when you set up a zone and, and want to start finding those patterns that may be invalid how do you go about doing that? What is the process?

Shawn Ambwani: 05:18 Yeah, so I mean there's two major things in the SEP zones that it's not, it's not just about finding invalid patents, although I can tell you it's relatively easy to find invalid patents in any of these zones. That's not a difficulty. The hard part is figuring out which ones to go to or which ones are going to be the most interesting to go after. And that takes a lot of art. Essentially identifying them, finding out good prior art that we feel comfortable with, hiring good counsel. There's all kinds of weighing mechanisms that go into it, who the entity is, how it came about, how old the patents are, where it came from. All of these variables go into that kind of equation of when we decide. What's kind of unique about the way we work is we work independently of our members, so our members are funding these activities and some join for free.

Shawn Ambwani: 06:11 So we have a number of members of the video codecs and we use all this money and information in our activities to basically go back and decide what to do, with the objective of deterring you know, what we consider bad assertions in this space. And then that's one part of it. The other part of it is that SEP's are all about, you know, an area called FRAND, fair, reasonable, non-discriminatory. And part of all of that involves negotiation. And so what we provide are tools to allow companies to negotiate we think in a fair and more transparent way to licensors as to pricing. But also explaining why the pricing is the way it is. Because one of the problems that we've had in the big picture is that a lot of these licensors have been asking for money, whether their own pools, whether outside of pools, whatever.

Shawn Ambwani: 07:12 But no one can really explain why the price is what it is. And I think that leads to a lot of people to just stop paying or stop wanting to get into licensing discussions. And that's not beneficial for the market. And so by explaining how the price comes out the way it is and providing a very, we consider, solid methodology for it, it allows our members but also licensors to better understand who owns what and how much value is in the standard. So what they should reasonably expect to get for that technology and how much licensees reasonably should expect to pay in order to deploy the technology.

Mark Donnigan: 07:55 Now my question, you know, Shawn is when you are getting into these conversations with the parties or party that, you know, owns this IP and I'm speaking more around sort of the pricing and the model and that sort of thing. Are you then...is that information available to your members or is it more that you're sort of helping facilitate, helping bring some rationality, you know, so that then that body can turn around and make public: "Hey, great news!" We've decided that all digitally distributed content doesn't carry, you know, a royalty cost. What exactly, I guess what my question is, what exactly is, is your role then in informing the market?

Shawn Ambwani: 08:40 I think that, well, I mean there's a number of things to talk about, but what's I think most important is that we, you know, we don't know necessarily what the right price is. We hired an outside economist to look into that and he came back with a pricing range in you know, a report that we gave the highlights too and there was some press over and it's on our website but you can also look at it through a number of particles and basically he came back with a price of between 8 cents and 28 cents I believe if I'm accurate. Is what he believed the estimate to be for the value of the technology including everything. And it ranged based on I think the device and like other factors and stuff like that. Now that high level information we provided publicly and in fact we provided the information on who made the report when it was created and what it was based on.

Shawn Ambwani: 09:38 And we even provided kind of the overall methodology of how it was done, which is basically being used at a very high level. They used MPEG LA's AVC license as the starting point or the foundation for deciding what HEVC in this case, which is what he was looking at, pricing should be based on his expert analysis. And then he modified that based on switching costs based on the cost of bandwidth, the cost of storage and quality and other factors basically that are valuable. So, that's where we went. Now, what's important to understand is that we published that information so anyone could take a look at the, at the high level. And the methodology pretty much tells you the roadmap of where we started and how we ended up where we are. The other part is how do you decide who you have to pay and how much each person gets. Even assuming that you figure out that, let's say it's 25 cents, that you think the royalty rate should be for it. And I'm not saying that's the number, but everyone can decide on whatever number they feel comfortable. Our expert created this report and we published it. Other people can create other reports and I'm sure they have their own kind of versions. But what's important for us is that, you know, people should explain why they came out with their pricing. And unfortunately in pools and licensing organizations in general, that just doesn't happen.

Dror Gill: 11:05 So basically you're finding economical reason behind a certain price for for this technology. In this case, HEVC. And now companies who want to use HEVC, how do they use this number? Because they have your number, which is the total, and then they have royalty rates that are asked that, you know, certain patent pools are asking and they add up to a different number that could be a higher number. So do they just you know, divide the number that they think is the right one among the different patent pools and pay them the amount they think they should pay or do they just use it as a negotiating tool when they talk to them and, and you know, and negotiate the actual world, the rates that they will have to pay?

Shawn Ambwani: 11:52 By providing a lot of this information. Some of it publicly like economic report in some format. The hope is that smaller entities instead of rolling over when licensing people come by and say, Hey, take it or leave it, they really have an ability to make a fair response, a good faith response with information that allows them to then basically justify why they came up with a price and really push back and say, listen, you know, this is what my methodology came out to. Now. It could be right, could be wrong. You know, in the end in FRAND negotiations, I have to make a good faith offer. That's really the intent. So that's an important aspect of pushing back on this kind of, we think less information that is occurring in the marketplace and more fragmentation. And I think they're all interrelated because of the less information you have more fragmentation. Cause if everyone could agree on a price and everyone agreed that this is the fair value for the technology, there really wouldn't be multiple pools in my opinion or multiple licensors, because everyone would know what the number is. And so why would you separate?

Dror Gill: 13:07 But basically you're saying that even if a patent pool set, the royalty rates and those royalty rates in some cases are public, at least for some of the patent pools, this is not what a licensor would pay. This is just kind of a starting point for a negotiation and you're providing tools for this type of negotiation.

Shawn Ambwani: 13:23 We also think that validity is a big issue because none of these entities look at validity when they're incorporating patents into their pools or into their licensing. It's really up to the licensees or the people who are potentially taking the license to have the responsibility to go out and figure that out, which can be very, very costly.

Dror Gill: 13:44 You assume they're valid, right? If they're licensing patents to you, you assume that they're licensing valid patents. Right? These are kind of, you know, respectable patent holders and patent pools. Why would they license something that's not valid?

Shawn Ambwani: 13:57 I mean, it's a great point. I mean, the argument would be that they want to license patents.

Mark Donnigan: 14:03 That's their business at the end of the day. Yeah.

Shawn Ambwani: 14:08 Right. So, you know, if you had a car and you're trying to sell a car, you're going to accentuate the good things about the car. Not that it's a rebuilt or something like that or you know, like it's, it's been, you know, it's been in a crash or accident like, yeah. Like you're going to show what you want to show. Right. And that's natural in any of these cases. The unfortunate fact is that it's very costly to figure out that stuff and there's no really organization you'd think a licensing organization like MPEG LA or others. And I'm not saying MPEG LA is doing a bad job necessarily, I'm just pointing them out as an example, would do a better job of vetting to some degree on that type of activity. But they don't, and I think there's a number of reasons

Mark Donnigan: 14:53 Why do they want to do that? I almost liken this to the 500 channel cable bundle of which there's about 15 high quality channels and there's 485 that are anywhere from just a, you know, not, not relevant, not interesting to, you know, to even lower quality than that, but, but you know, but Hey, I got a 500 channel bundle, right? So I feel like, wow, it must be worth $100 a month, you know, or whatever.

Shawn Ambwani: 15:23 The idea that that licensing organizations like MPEG LA or (HEVC) Advance or other ones like that aren't doing it to the benefit of their licensors. It just seems ridiculous to me. I mean the people on their, on their management and the people who are actually owning that organization, typically it's managed and owned and administrative fees are paid to licensors. And traditionally the money flows one way from licensees to licensors. It's for the benefit of the licensors. And the rules that they put in are essentially to make sure that those guys are protected. They have no incentive in general of saying people's patents are invalid. And, and that's just a bad fact pattern for them. If basically they get back and say, Hey, listen this patent... Yeah, no, it's bad.

Mark Donnigan: 16:16 Exactly. So, so in that context then it completely makes sense that they don't vet you know, at the level that you are and why, you know, Unified Patents needs to exist, you know, is because we need this sort of independent third party. I guess. I, I, you know, that's, that's out there doing this work. Now, Shawn, one of the things that I noticed is you're acting both against NPE's, so, non-practicing entities, and against SEP's. So standard essential patents. What are the issues with SEP's?

Shawn Ambwani: 16:51 Well, I mean the general assumption has been, and I don't know where this assumption came from, was that standard essential patents or people who declare their patents to be standard essential are more likely to be valid than other patents. And in the real world where there's litigation and there's challenges and things get checked out or vetted essentially, adversarially, the reality is that standard essential patents in all the studies that have been done fair, far worse, than normal patents do on average. And you know, it's not shocking actually when you think about it. Obviously there's a lot of self selection here, but part of the reason why is, you know, when you're submitting into pools or in when you're getting these patents, when part of a standardization body or doing other activities, there's a lot of other people involved and it's usually built on other ideas that people have had in the past.

Shawn Ambwani: 17:59 And it's not surprising that a lot of these patents have underlying ideas that had been done in the past or other people had brought up previously. Sometimes they weren't accepted, sometimes they were or sometimes they were put on hold. Who knows? But there's a lot of prior art oftentimes in these areas. These aren't open fields, these aren't brand new innovations that typically come up. And so that's not surprising. Now, you know, there's also a general belief that standard essential patterns are more valuable. And I think, you know, that's a pretty, I would say, you know, I dunno if it's absolutely valid, but it's not unreasonable to believe that if you declare a patent, as standardized, if you look at the average patent and compared to that patent, it's probably your, it's probably more valuable, at that point. Because you basically said it's part of a standard that people are probably going to adopt at that point versus a patent in general, which you never know most of the time, whether anyone's going to use that patent.

Shawn Ambwani: 19:02 I mean the vast majority of patents are never actually used in any way whatsoever. They're not enforceable because they're just ideas that people have most of the time, and these patents are arguable more likely than not to be in a standard and that standard might or might not actually get used in the end. Inherently you get - they're more valuable. The problem is there's tons of over declaration that occurs in this area. There's very little incentive. I mean some places there's more of an incentive than others, but the way MPEG works specifically is that you can do blanket declarations and so you don't have to declare specific patents. And, other standards, you have to basically declare each individual patent that you have. So, I mean, there's all kinds of trade offs, and all these different things, but the reality is that no one really knows exactly how many patents need to be licensed. And that just creates a lot of uncertainty. And you know, a lot of companies who are trying to make money, not off products but off of doing licensing thrive on uncertainty because that's where they can make money. Is basically by, you know, saying, okay, well who knows what can happen, but if you take care of me now, I can make sure that I'm not going to cause you issues.

Dror Gill: 20:23 Right. And that's why uncertainty is in the middle of FUD, fear, uncertainty, doubt, which is one of those tactics and uncertainty is definitely a big part of that.

Shawn Ambwani: 20:33 Yeah. I mean, the other thing is that companies in general, it seems like a one way street a lot of the time, which is pretty unfortunate in that although I'm not sure if I have a good solution, you know, a lot of companies, the licensors have a way of getting together, agreeing on a price and then licensing through an organization like MPEG LA or others to do that type of activity or Velos (Media), or whatever it is. They choose, you know, they can select a price, they can work together, agree on a price. And the reason why they can do that according to the DOJ is because it's a different product than what's available before. So it decreases uncertainty by making it easier for people to take a license of convenience for that specific technology area.

Dror Gill: 21:21 Otherwise, it might might've been considered price setting,

Shawn Ambwani: 21:24 Right? Yeah. It would be considered price. It would be considered price setting. But in this case, the argument is always that you can always go to each individual company and get a license or negotiate a separate license. This is a license of convenience for this technology area from all these companies for one price. And that makes it a lot easier for people on both sides to be able to know exactly how much they're going to be getting and how much they're going to have to pay for clearing this risk. Which makes sense. I fundamentally have no problem with pools and what they do. The, the issue comes up is that a lot of these pools, A) don't talk about the pricing, they don't look at the validity. They don't really have a great essentially checking on top of it. And they're very much incentivized to help out the licensors, not the licensees figure stuff out. And what ends up happening is over time you kind of, and you have companies also that are not interested in making products, which is unfortunate. They're just interested in making money off of their licensing. Which is unfortunate because there's a lot of games that can be played in the standardization world to get your stuff in and then get your patents in basically.

Mark Donnigan: 22:44 Well, it ultimately, it, it stops innovation. I mean, at the end of the day, you know, and one thing, and Dror and I have talked about this on episodes and we've certainly talked about this a lot, you know, privately within Beamr is, you know, it's a little bit mystifying as well because okay, so HEVC clearly was set back as a result of, of many issues. But you know, largely what we'd been talking about for the last 35 minutes and the adoption of HEVC. And yet these people, as you point out, the licensors, they don't make money if nobody's using the technology. So, so what's mystifying to us is that this is not, you know, it's not like somehow they're getting paid still. You know, even though the adoption of the technology is not in place or it's not being used, they're not getting paid. And so it seems like at some point, you know, a rational actor would stand up and say, wait a second, I'd rather get something rather than nothing! But, it's almost like they, they're not acting that way.

Dror Gill: 23:46 But, but it did happen. They did reduce the royalty rate. Yes, yes, yes. Certainly. And they did come to their senses and they did put a cap and then initially it was uncapped and they did remove royalties from content. And you know, they did a lot of things in the right direction after the pressure from the market when they realized they're not going to get anything. And when AV1 started to happen, you know, and they were pressured by that, by a competing codec that was supposedly a royalty free and didn't have these issues. So I think the situation is improved. But you've launched a specific zone. It's called the video codec zone, but basically right now it deals only with HEVC.

Shawn Ambwani: 24:33 A lot of these patents that we've challenged relate not just to HEVC but potentially to AV and other codecs like AVC as well. Cause there's such overlap between these things. That's why we generically call it a video codec zone. So, obviously a lot of the things that we've looked at in like the economic report and everything else and landscape, a lot of the focus has been on HEVC.

Dror Gill: 24:59 So you examined the HEVC and and you saw this situation that you have three patent pools. One of them hasn't even announced the royalty rates and, and you have a lot of independent patent holders who claim to have standard essential patents for HEVC. And this is kind of your, you're opening a, a situation. So what, what was the first thing that, that you did, how did you start to, to approach the HEVC pattern topic and what actions did you take?

Shawn Ambwani: 25:34 Like I said, we've done a bunch of different stuff. We had a submission repo called open, which where we collated all the prior art, not prior art, but submissions into the standard for HEVC and AVC and other standards from MPEG so people can make it easily searchable. In fact, 50% of the priority art that we got for our patent challenges came from the submission repo, which is great, which is basically, you know, previous submissions to the same standard. We have OPAL, which is our landscape tool. And then, you know, obviously we have OPEN which is our evaluation report that I mentioned for HEVC. And then we did a bunch of reviews of validity and challenged a bunch of patents in different licensing entities. I mean, Velos, I think they don't consider themselves a pool. Just to be clear.

Dror Gill: 26:29 Because they actually own the patents. They've licensed those patents on their own?

Shawn Ambwani: 26:34 Well, I think they just don't consider themselves tackling a patent pool in the way that MPEG LA and HEVC Advanced does simply that would throw them into a different bucket and they would have all kinds of requirements on them that they don't want basically. So you know when the DOJ kind of made the rules or kind of the lawyers decided what the right rules are to make it work, you know like you've got to show your stuff. Basically you got to show your price, you've got to make sure it's reasonable or it's, you know, like there's, there's no most favored nation clause. I mean there is a most favorite nation (MFN) let me rephrase this. So all these things to make sure that everything is very transparent in order to allow this kind of companies to get together and set a price for how much they want to license for it, which typically would have huge anti-competitive or antitrust issues. Right. They made all these rules and Velos I think would not consider themselves technically a patent pool like those guys because that would make them have the similar requirements.

Dror Gill: 27:40 So they're like an independent patent holder?

Shawn Ambwani: 27:42 I don't really know what they call themselves. I've definitely never heard them say that they're a patent pool. I've heard other people call them a patent pool. I probably have at some point, but I don't really know if they actually consider themselves a patent pool

Dror Gill: 27:55 Because I noticed that your litigation was against the patent holders. Companies like GE and KBS and against Velos Media itself. Yeah.

Shawn Ambwani: 28:07 Yeah. Well Velos is you know, an unusual beast in that it owns a number of patents that got transferred to it as well as it provides licenses to the people who participated. You know, the other patent holders in general are much more traditional in their patent pool type activity in that the patent holders are different from the people who are doing the licensing.

Dror Gill: 28:28 And you're not suing the patent pools like MPEG LA and HEVC Advanced or not your targets?

Shawn Ambwani: 28:32 Well, they don't own patents directly, so really nothing to do as far as I know. I mean, you could say, you know, part of it is we're challenging them to a certain degree on their pricing and kind of their whole model of not looking at validity by challenging some of their patents as well as, you know, putting them on notice that as they get more patents in, we might challenge further patents for validity. So why don't they do it ahead of time? I mean, the idea that, you know, validity is a victimless crime if you don't check for validity, it doesn't hurt anyone. It's just not true in my opinion. It's just not true because you are hurting the people who actually innovated. There's a set amount of money that goes to everyone. If you have a bunch of patents and they're just like, you're checking for essentiality before you allow a patent in, you check for validity because there's a bunch of patents that just aren't valid that shouldn't be, they should not be making money off of. It just incentivizes people to get more invalid patents in the same space that they can stick into a pool to get a bigger share of it, like a giant game. Right?

Mark Donnigan: 29:43 Yeah, that's a really good point. I'm wondering what is the cost to test for essentiality? Is some of this just sort of practical like it's just either too time consuming or costly to test? Yeah. I mean

Shawn Ambwani: 29:58 Esentiality is often times more expensive than validity in some cases, but I mean they do test for essentiality. The companies pay to have their own patents tested often times for essentiality, but there is no test for validity that they enforce. So no one actually does it. You know, if they did ask for it, I'm sure people in some cases would pay for it, but more importantly, people who didn't think their patents would be found valid, probably wouldn't submit them in the first place then. Then there would be, there'd be huge disincentive for people who had that risk of that happening. They just wouldn't submit it, which you know, obviously it's going to hurt the pool because they get less patents. And at the same time, the hope is that people will think twice before they submit stuff they know is crap. Anyway.

Mark Donnigan: 30:43 So what is, what is your bar for determining low quality? I mean, what does that process look like?

Shawn Ambwani: 30:52 We have a bunch of patents that come into our hopper that we're constantly looking at in every single zone that we're in and we're constantly looking and seeing if it's a valid patent or not. And there's multiple ways of doing that. We have crowd sourcing that we do for that. We just pay people, you know, in order to do prior art experts for example, to do prior art searches. You can prior art search infinitely long these, there's no stopping. You know, what you can do. But you know, in the end there's only so much you can reasonably expect to find. And so from my perspective, you know, there's definitely been situations where we've looked at patents and we've said, okay, we don't think we have good prior art. We're just not going to do anything about it. And that's okay. In fact, I mean it's okay if a licensing entity or licensor has a valid patent, that's perfectly fine with me.

Shawn Ambwani: 31:49 I think if they have a valid patent, they should be able to make money off of it. I have no problem with, it should be a fair amount if it's in a standard based on FRAND principles, but in general, people should be able to make money off of a valid patent. The problem is is that a lot of people are making a lot of money, in my opinion, off of a lot of bad patents. And invalid patents, which hurts the people who actually do have good patents because they're getting crowded out, which is sad because that really is the disincentive for innovation then is when the people actually are innovating aren't making money off of it because they're getting crowded out by the people who are just playing a good game.

Dror Gill: 32:25 You described earlier the, the process with a standard setting bodies such as MPEG where you declare your patents but you only, you can declare them as, as a pool or as a bunch of patents and not specifically, and then you can basically,

Dror Gill: 32:40 You know, create a pool and charge as much as you want if it's under the FRAND principles. Do you think there's anything broken in the standard setting process itself? Do those committees need to do something else in order to make sure that when they create a standard, the situation of royalties of, of the situation of, of IP which is essential to that standard is more well known that you have less uncertainty in that IP?

Shawn Ambwani: 33:09 Yeah, I'm not sure. I mean there's always ways of like tweaking the system. Every standards body has different ways of managing it. I mean the only really clean way of doing it is saying it's royalty free and having anyone who participates in the standard agree that it's royalty free. Anything above that, just you know, you can play all these different types of rules and machinations and 3GPP has their own and other people, organizations have their own. But in the end it ends up being the same issue of you know, under declaring over declaring - issues with essentiality, validity, all kinds of other things. So I'm not sure if you, unless you go to that binary level, how much, you know, changing that up is going to change things fundamentally. I think the more fundamental thing is that, you know, the idea that I think the fundamental reason why you have these patent pools and other things like that was to clear risk and decrease uncertainty. Unfortunately I'd say uncertainty is actually increasing in some of these cases not decreasing by all these different groups asking for money at this point, which is unfortunate.

Dror Gill: 34:17 No, that's a very interesting insight, really.

Mark Donnigan: 34:19 Hey, thanks for joining us, Shawn. This was really an amazing discussion and we definitely have to have a part two.

Shawn Ambwani: 34:26 All right, well, thanks for your time, gentlemen. I really appreciate it.

Dror Gill: 34:28 Thank you.

Narrator: 34:30 Thank you for listening to The Video Insiders podcast, a production of Beamr Imaging limited. To begin using Beamr's codecs today, go to beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.

Resources:

Download HEVC deployment statistics document here: JCTVC-AK0020

Related episode: E08 with MPEG Chairman Leonardo Charliogne

The Video Insiders LinkedIn Group is where we host engaging conversations with over 1,500 of your peers. Click here to join

Like to be a guest on the show? We want to hear from you! Send an email to: thevideoinsiders@beamr.com

Learn more about Beamr's technology

Rob Koenen, Co-Founder of Tiledmedia, discusses the latest advancements in HEVC VR encoding with The Video Insiders. You will learn about video encoding issues relating to HEVC tile encoding, 8K, MP4 metadata optimization for high-resolution files, and more.

Join the conversation by jumping into The Video Insiders LinkedIn Group.

If you would like to be a guest on the show, send an email to thevideoinsiders@beamr.com.

For more podcast episodes, click here.

Learn more about Beamr's technology.

Today's guest: Rob Koenen.

E04: In this episode, The Video Insider's catch up with industry expert, Tim Siglin, to discuss HEVC implementation trends that counter previous assumptions, notable 2018 streaming events, and what's coming in 2019.

The following blog post first appeared on the Beamr blog at: https://blog.beamr.com/2019/01/01/2018-the-year-hevc-took-flight/

By now, most of us have seen the data and know that online video consumption is soaring at a rate that is historically unrivaled. It’s no surprise that in the crux of the streaming era, so many companies are looking to innovate and figure out how to make their workflows or customers workflows better, less expensive, and faster.

In Episode 4 of The Video Insiders, we caught up with streaming veteran Tim Siglin to discuss HEVC implementation trends that counter previous assumptions, notable 2018 streaming events, and what’s coming in 2019.
Tune in to hear The Video Insiders cover top-of-mind topics:

HEVC for lower resolutions
Streaming the World Cup
Moving from digital broadcast to IP-based infrastructure
What consumers aren’t thinking about when it comes to 4K and HDR
Looking forward into 2019 & beyond
Tune in to Episode 04: 2018, the Year HEVC Took Flight or watch the video below.

Want to join the conversation? Reach out to TheVideoInsiders@beamr.com

TRANSCRIPTION (lightly edited to improve readability only)

Mark Donnigan: 00:00 On today’s episode, the Video Insiders sit down with an industry luminary who shares results of a codec implementation study, while discussing notable streaming events that took place in 2018 and what’s on the horizon for 2019. Stay tuned. You don’t want to miss receiving the inside scoop on all this and more.

Announcer: 00:22 The Video Insiders is the show that makes sense of all that is happening in the world of online video, as seen through the eyes of a second generation Kodak nerd and a marketing guy who knows what I frames and macroblocks are. Here are your hosts, Mark Donnigan and Dror Gill.

Mark Donnigan: 00:40 Welcome, everyone. I am Mark Donnigan, and I want to say how honored Dror and I are to have you with us. Before I introduce this very special guest and episode, I want to give a shout of thanks for all of the support that we’re receiving. It’s really been amazing.

Dror Gill: 00:58 Yeah. Yeah, it’s been awesome.

Mark Donnigan: 00:59 In the first 48 hours, we received 180 downloads. It’s pretty amazing.

Dror Gill: 01:06 Yeah. Yeah, it is. The industry is not that large, and I think it’s really an amazing number that they’re already listening to the show from the start before the word of mouth starts coming out, and people spread the news and things like that. We really appreciate it. So, if it’s you that is listening, thank you very much.

Mark Donnigan: 01:29 We really do aim for this to be an agenda-free zone. I guess we can put it that way. Obviously, this show is sponsored by Beamr, and we have a certain point of view on things, but the point is, we observed there wasn’t a good place to find out what’s going on in the industry and sort of get unbiased, or maybe it’s better to say unfiltered, information. That’s what we aim to do in every episode.

Mark Donnigan: 01:57 In this one, we’re going to do just that. We have someone who you can definitely trust to know what’s really happening in the streaming video space, and I know he has some juicy insights to share with us. So, without further ado, let’s bring on Tim Siglin.

Tim Siglin: 02:15 Hey, guys. Thank you for having me today and I will definitely try to be either as unfiltered or unbiased as possible.

Mark Donnigan: 02:21 Why don’t you give us a highlight reel, so to speak, of what you’ve done in the industry and, even more specifically, what are you working on today?

Tim Siglin: 02:31 Sure. I have been in streaming now for a little over 20 years. In fact, when Eric Schumacher-Rasmussen came on as the editor at StreamingMedia.com, he said, “You seemed to be one of the few people who were there in the early days.” It’s true. I actually had the honor of writing the 10-year anniversary of Streaming Media articles for the magazine, and then did the 20-year last year.

Tim Siglin: 02:57 My background was Motion Picture production and then I got into video conferencing. As part of video conferencing, we were trying to figure out how to include hundreds of people in a video conference, but not need necessarily have them have two-way feedback. That’s where streaming sort of caught my eye, because, ultimately, for video conferencing we maybe needed 10 subject matter experts who would talk back and forth, and together a hundred, then it went to thousands, and now hundreds of thousands. You can listen in and use something like chat or polling to provide feedback.

Tim Siglin: 03:31 For me, the industry went from the early revolutionary days of “Hey, let’s change everything. Let’s get rid of TV. Let’s do broadcast across IP.” That was the mantra in the early days. Now, of course, where we are is sort of, I would say, two-thirds of the way there, and we can talk a little bit about that later. The reality is that the old mediums are actually morphing to allow themselves to do heap, which is good, to compete with over the top.

Tim Siglin: 04:01 Ultimately, what I think we’ll find, especially when we get to pure IP broadcast with ATSC 3.0 and some other things for over-the-air, is that we will have more mediums to consume on rather than fewer. I remember the early format ways and of course we’re going to talk some in this episode about some of the newer codec like HEVC. Ultimately, it seems like the industry goes through the cycles of player wars, format wars, browser wars, operating system wars, and we hit brief periods of stability which we’ve done with AVC or H.264 over the last probably eight years.

Tim Siglin: 04:46 Then somebody wants to stir the pot, figure out how to either do it better, less expensively, faster. We go back into a cycle of trying to decide what the next big thing will be. In terms of what I’m working on now, because I’ve been in the industry for almost 21 years. Last year, I helped start a not-for-profit called Help Me Stream, which focuses on working with NGOs in emerging economies, trying to help them actually get into the streaming game to get their critical messages out.

Tim Siglin: 05:18 That might be emerging economies like African economies, South America, and just the idea that we in the first world have streaming down cold, but there are a lot of messages that need to get out in emerging economies and emerging markets that they don’t necessarily have the expertise to do. My work is to tie experts here with need there and figure out which technologies and services would be the most appropriate and most cost effective.

Mark Donnigan: 05:46 That’s fascinating, Tim.

Tim Siglin: 05:48 The other thing I’m working on here, just briefly, is we’re getting ready for the Streaming Media Sourcebook, the 2019 sourcebook. I’m having to step back for the next 15 days, take a really wide look at the industry and figure out what the state of affairs are.

Dror Gill: 06:06 That’s wonderful. I think because this is exactly the right point, is one you end and the other one begins, kind of to summarize where we’ve been in 2018, what is the state of the industry and the fact that you’re doing that for the sourcebook, I think, ties in very nicely with our desire to hear from you an overview of what were the major milestones or advancements that were made in the streaming industry in 2018, and then looking into next year.

Dror Gill: 06:39 Obviously, the move to IP, getting stronger and stronger, now the third phase after analog and digital, now we have broadcast over IP. It’s interesting what you said about broadcasters not giving up the first with the pure OTT content providers. They have a huge business. They need to keep their subscribers and lower their churn and keep people from cutting the cord, so to speak.

Dror Gill: 07:04 The telcos and the cable companies still need to provide the infrastructure for Internet on top of which the over-the-top providers and their content, but they still need to have more offering and television and VLD content in order to keep their subscribers. It’s very interesting to hear how they’re doing it and how they are upgrading themselves to the era of IP.

Tim Siglin: 07:30 I think, Dror, you hit a really major point, which is we, the heavy lift … I just finished an article in ATSC 3.0 where I talk about using 2019 to prepare for 2020 when that will go live in the U.S. The heavy lift was the analog to digital conversion. The slightly easier lift is the conversion from digital to IP, but it still requires significant infrastructure upgrade and even transmission equipment to be able to do it correctly for the over-the-year broadcasters and cable.

Dror Gill: 08:07 That’s right. I think on the other hand, there is one big advantage to broadcast, even broadcast over-the-air. That is the ability to actually broadcast, the ability to reach millions, tens of millions, hundreds of millions of people over a single channel that everybody is receiving. Whereas, because of historic reasons and legacy reasons in IP, we are limited, still, when you broadcast to the end user to doing everything over unicast. When you do this, it creates a tremendous load on your network. You need to manage your CDNs.

Dror Gill: 08:46 I think we’ve witnessed in 2018 on one hand very large events being streamed to our record audience. But, on the other hand, some of them really failed in terms of user experience. It wasn’t what they expected because of the high volume of users, and because more and more people have discovered the ability to stream things over IP to their televisions and mobile devices. Can you share with us some of the experience that you have, some of the things that you’re hearing about in terms of these big events where they had failures and what were the reasons for those failures?

Tim Siglin: 09:30 I want to reiterate the point you made on the OTA broadcast. It’s almost as if you have read the advanced copy of my article, which I know you haven’t because it’s only gone to the editor.

Dror Gill: 09:42 I don’t have any inside information. I have to say, even though we are the Video Insiders.

Mark Donnigan: 09:47 We are the Video Insiders. That’s right.

Dror Gill: 09:49 We are the Video Insiders, but …

Mark Donnigan: 09:49 But no inside information here.

Dror Gill: 09:51 No inside information. I did not steal that copy.

Tim Siglin: 09:55 What I point out in that article, Dror, I think which will come out in January shortly after CES is basically this. We have done a good job in the streaming industry, the OTT space of pushing the traditional mediums to upgrade themselves. One of the things as you say with OTA, that ability to do essentially a multicast from a tower wirelessly is a really, really good thing, because to get us to scale, and I think about things like the World Cup, the Olympics and even the presidential funeral that’s happened here in December, there are large-scale events that we in the OTT space just can’t handle, if you’re talking about having to build the capacity.

Tim Siglin: 10:39 The irony is, one good ATSC transmission tower could hit as many people as we could handle essentially globally with the unicast (OTT) model. If you look at things like that and then you look at things like EMBMS in the mobile world, where there is that attempt to do essentially a multicast, and it goes to points like the World Cup. I think one of the horror stories in the World Cup was in Australia. There was a mobile provider named Optus who won the rights to actually do half of the World Cup preliminary games. In the first several days, they were so overwhelmed by the number of users who wanted to watch and were watching, as you say, in a unicast model that they ended up having to go back to the company they had bid against who had the other half of the preliminaries and ask them to carry those on traditional television.

Tim Siglin: 11:41 The CEO admitted that it was such a spectacular failure that it damaged the brand of the mobile provider. Instead of the name Optus being used, everybody was referring to it as “Floptus.” You don’t want your brand being known as the butt of jokes for an event that only happens once every four years that you have a number of devotees in your market. And heaven forbid, it had been the World Cup for cricket, there would have been riots in the street in Sydney and Melbourne. Thank goodness it was Australia with soccer as opposed to Australia with cricket.

Tim Siglin: 12:18 It brings home the point that we talk about scale, but it’s really hard to get to scale in a unicast environment. The other event, this one happened, I believe, in late 2017, was the Mayweather fight that was a large pay-per-view event that was streamed. It turned out the problem there wasn’t as much the streams as it was the authentication servers were overwhelmed in the first five minutes of the fight. So, with authentication gone, it took down the ability to actually watch the stream.

Tim Siglin: 12:53 For us, it’s not just about the video portion of it, it’s actually about the total ecosystem and who you’re delivering to, whether you’re going to force caps into place because you know you can’t go beyond a certain capacity, or whether you’re going to have to partner up with traditional media like cable service providers or over-the-air broadcasters.

Mark Donnigan: 13:14 It’s a really good point, Tim. In the World Cup, the coverage that I saw, it was more of, I’d almost say or use the phrase, dashed expectations. Consumers, they were able to watch it. In most cases, I think it played smoothly. In other words, the video was there, but HDR signaling didn’t work or didn’t work right. Then it looked odd on some televisions or …

Tim Siglin: 13:40 In high frame rate …

Tim Siglin: 13:43 20 frames a second instead of 60 frames a second.

Mark Donnigan: 13:48 Exactly. What’s interesting to me is that, what I see is, the consumer, they’re not of course walking around thinking as we are, like frame rate and color space and resolution. They are getting increasingly sensitive to where they can look at video now and say, “That’s good video,” or “That doesn’t look right to me.” I know we were talking before we started recording about this latest Tom Cruise public service announcement, which is just super fascinating, because it …

Tim Siglin: 14:24 To hear him say motion interpolation.

Mark Donnigan: 14:26 Yeah. Maybe we should tell the audience, for those, since it literally just came out I think today, even. But you want to tell the audience what Tom Cruise is saying?

Tim Siglin: 14:38 Essentially, Tom Cruise was on the set of Top Gun, as they’re shooting Top Gun. Another gentleman did a brief PSA for about a minute asking people to turn off motion interpolation on their televisions, which motion interpolation essentially takes a 24-frame per second and converts it to 30 frames per second by adding phantom frames in the middle. Because Mission Impossible: Fallout is just being released for streaming, Cruise was concerned and obviously others were concerned that some of the scenes would not look nearly as good with motion interpolation turned on.

Tim Siglin: 15:17 I think, Mark, we ought to go to a PSA model, asking for very particular things like, “How do you turn HDR on? How do you …” Those types of things, because those get attention in a way that you and I or a video engineer can’t get that attention.

Dror Gill: 15:33 How do you know if what you’re getting is actually 4K or interpolate HD, for example?

Tim Siglin: 15:38 Especially in our part of the industry, because we will call something OTT 4K streaming. That may mean that it fits in a 4K frame, but it doesn’t necessarily mean it’s that number of pixels being delivered.

Dror Gill: 15:52 It can also mean that the top layer in your adaptive bit rate stream is 4K, but then if you don’t have enough bandwidth, you’re actually getting the HD layer or even lower.

Tim Siglin: 16:01 Exactly.

Dror Gill: 16:02 Even though it is a 4K broadcast and it is 4K content. Sometimes, you can be disappointed by that fact as well.

Mark Donnigan: 16:11 I have to give a very, very funny story directly related, and this happened probably, I don’t know, maybe, at least 18 months ago, maybe two years ago. I’m sitting on an airplane next to this guy. It’s the usual five-minute, get acquainted before we both turn on our computers. Anyway, when someone asks, “What do you do?” I generally just say, “I work for a video software company,” because how do you explain digital encoding? Most people just sort of stop at that, and don’t really ask more.

Mark Donnigan: 16:44 But this guy is like, “Oh, really?” He said, “So, I just bought a 4K TV and I love it.” He was raving about his new Samsung TV. Of course, he figured I’m a video guy. I would appreciate that. I said, “Hey.” “So, you must subscribe to Netflix.” “Yes. Yes, of course,” he says. I said, “What do you think of the Netflix quality? It looks great, doesn’t it?”

Mark Donnigan: 17:10 He sort of hem and hawed. He’s like, “Well, it really … I mean, yeah. Yeah, it looks great, but it’s not quite … I’m just not sure.” Then, I said, “I’m going to ask you two questions. First of all, are you subscribed to the 4K plan?” He was. Then I said, “How fast is your Internet at home.” He’s like, “I just have the minimum. I don’t know. I think it’s the 20 megabit package,” or whatever it was. I don’t remember the numbers.

Mark Donnigan: 17:38 I said, “There’s this thing.” And I gave him like a 30-second primer on adaptive bit rate, and I said, “It is possible, I have no idea of your situation, that you might be watching the HD version.” Anyway, he’s like, “Hah, that’s interesting.” I connect with the guy on LinkedIn. Three days later, I get this message. He says, “I just upgraded my Internet. I now have 4K on my TV. It looks awesome.”

Mark Donnigan: 18:04 On one hand, the whole situation was not surprising and, yet, how many thousands, tens of thousands, maybe millions of people are in the exact same boat? They’ve got this beautiful TV. It could be because they’re running some low-end router in the house. It could be they truly have a low end bandwidth package. There could be a lot of reasons why they’re not getting the bandwidth. They’re so excited about their 4K TV. They’re paying Netflix to get the top layer, the best quality, and they’re not even seeing it. It’s such a pity.

Tim Siglin: 18:37 I had a TSA agent asked me that same question, Mark, when I came through customs. I’m like, “Sure. I’ll stand here and answer that question for you.” The router was actually what I suggested that he upgrade, because he said his router was like this (old unit).

Mark Donnigan: 18:53 In a lot of homes, it’s a router that’s 15 years old and it just isn’t (up to the task).

Tim Siglin: 18:58 But it brings out the point that even as we’re talking about newer codecs and better quality, even if we get a lower sweet spot in terms of 4K content (streaming bandwidth), or as we found in the survey that we worked on together, that using HEVC for 1080p or 720p, if the routers, if the software in the chain is not updated, the delivery quality will suffer in a way that people who have a tuned television and seen the consistent quality aren’t certain what to do to fix when they use an over-the-top service.

Tim Siglin: 19:34 I think this is a key for 2019. As we prepare for ATSC 3.0 on over-the-air broadcast where people will be able to see pristine 4K, it will actually force those of us in the OTT space to up our game to make sure that we’re figuring out how to deliver across these multiple steps in a process that we don’t break.

Dror Gill: 19:54 You really see ATSC 3.0 as a game-changer in 2019?

Tim Siglin: 19:59 What I see it as is the response from the broadcast industry to, A) say that they’re still relevant, which I think is a good political move. And, B) it provides the scale you were talking about, Dror. See, I think what it does is it at least puts us in the OTT space on notice that there will be in certain first world countries a really decent quality delivery free of charge with commercials over the air.

Tim Siglin: 20:31 It takes me back to the early days of video compression when, if you had a good class-one engineer and an analog NTSC transmission system, they could give you really good quality if your TV was tuned correctly. It only meant having to tune your TV. It didn’t mean having to tune your router or having to tune your cable modem, having to tune your settings on your TV. I think that’s where the game-changer may be, is that those tuner cards, which will send HDR signaling and things like that with the actual transmission, are going to make it much easier for the consumer to consume quality in a free scenario. I think that part of it is a potential game-changer.

Mark Donnigan: 21:19 That’s interesting. Tim, we worked together earlier this year on a survey, an industry survey that I think it would be really, really interesting to listeners to talk about. Shall we pivot into that? Maybe you can share some of the findings there.

Tim Siglin: 21:38 Why don’t you take the lead on why Beamr wanted to do that? Then I’ll follow up with some of the points that we got out of it.

Mark Donnigan: 21:46 Obviously, we are a codec developer. It’s important for us to always be addressing the market the way that the market wants to be addressed, meaning that we’re developing technologies and solutions and standards that’s going to be adopted. Clearly, there has been, especially if we rewind a year ago or even 18 months ago, AV1 was just recently launched. There were still questions about VP9.

Mark Donnigan: 22:19 Obviously, H264 AVC is the standard, used everywhere. We felt, “Let’s go out to the industry. Let’s really find out what the attitudes are, what the thinking is, what’s going on ‘behind closed doors’ and find out what are people doing.” Are they building workflows for these new advanced codecs? How are they going to build those workflows? That was the impetus, if you will, for it.

Mark Donnigan: 22:49 We are very happy, Tim, to work with you on that and of course Streaming Media assisted us with promoting it. That was the reason we did it. I know there were some findings that were pretty predictable, shall we say, no surprises, but there were some things that I think were maybe a little more surprising. So, maybe if you like to share some of those.

Tim Siglin: 23:12 Yeah. I’ll hit the highlights on that. Let me say too that one of the things that I really like about this particular survey, there was another survey that had gone on right around that time that essentially was, “Are you going to adopt HEVC?” What we took the approach on with this survey was to say, “Okay. Those of you who’ve already adopted HEVC, what are the lessons that we can learn from that?”

Tim Siglin: 23:36 We didn’t exclude those who were looking at AV1 or some of the other codes, even VP9, but we wanted to know those people who used HEVC. Were they using it in pilot projects? Were they thinking about it? Were they using it in actual production? What we found in the survey is that AVC, or H.264, was still clearly dominant in the industry, but that the ramp-up to HEVC was moving along much faster than at least I … I believed. Mark, I told you when we started the survey question creation, which was about a year ago and then launched it in early 2018, I expected we wouldn’t see a whole lot of people using HEVC in production.

Tim Siglin: 24:23 I was pleasantly surprised to say that I was wrong. In fact, I think you mentioned in our recent Streaming Media West interview that there was a statistic you gave about the number of households that could consume HEVC. Was it north of 50%?

Mark Donnigan: 24:40 Yeah, it’s more than 50%. What’s interesting about that number is that that actually came from a very large MSO. Of course, they have a very good understanding of what devices are on their network. They found that there was at least one device in at least 50% of their homes that could receive and decode, playback, HEVC. That’s about as real world as you can get.

Tim Siglin: 25:06 What was fascinating to me too in this study was, we asked open-ended questions, which is what I’ve done in the research projects for the last 25 years both the video conferencing and streaming. One of the questions we asked was, “Do you see HEVC as only a 4K solution or do you see it as an option for lower resolutions?” It turned out overwhelmingly, people said, “We not only see it for 4K. We see it for high-frame rate (HFR) 1080p, standard frame rate 1080p, with some HDR.”

Tim Siglin: 25:40 Not a majority, but a large number of respondents said they would even see it as a benefit at 720p. What that tells me is, because we had a large number of engineers, video engineers, and we also have people in business development who answer these questions, what it tells me is that companies know as we scale because of the unicast problem that Dror pointed out in the beginning that scaling with a codec that consumes more bandwidth is a good way to lose money, kind of like the joke that the way a rich man can lose money really fast is to invest in an airline.

Tim Siglin: 26:19 If indeed you get scale with AVC, you could find yourself with a really large bill. That look at HEVC is being not just for 4K, HDR, or high frame rate in the future, but also for 1080p with some HDR and high frame rate. It tells me that the codec itself or the promise of the codec itself was actually really good. What was even more fascinating to me was the number of companies that had AVC pipelines that were actually looking to integrate HEVC into those same production pipe.

Tim Siglin: 26:55 It was much easier from a process standpoint to integrate HEVC into an AVC pipeline, so in other words, H265 into H264 pipeline than it was to go out of house and look at something like AV1 or VP9, because the work that was done on HEVC builds on the benefits that were already in place in AVC. Of course, you got Apple who has HLS, HTTP Live Streaming, and a huge ecosystem in terms of iPhones and iPads, laptops and desktops supporting HEVC not just as a standard for video delivery, but also with the HEIC or HEIF image format, now having all of their devices shoot images using HEVC instead of JPEG. That in and of itself drives forward adoption of HEVC. I think you told me since that survey came out, probably now seven months ago, you all have continued to see the model of all-in HEVC adoption.

Dror Gill: 28:03 This is what we promote all the time. It’s kind of a movement. Are you all in HEVC or are you doing it just for 4K, just where you have to do it? We really believe in all-in HEVC. Actually, this week, I had an interesting discussion with one of our customers who is using our optimization product for VOD content, to reduce bit-rate of H.264 (streams). He said, “I want to have a product. I want to have a solution for reducing bit-rates on our live channels.”

Dror Gill: 28:32 So, I asked them, “Okay. Why don’t you just switch your codec to HEVC?” He said, “No, I can’t do that.” I said, “Why not?” He said, “You know compatibility and things like that.” I asked, “Okay. What are you using? What are you delivering to?” He said, “We have our own set-top boxes (STB), IP set-top boxes which we give out to our customers. Well, these are pretty new.” So, they support HEVC. I’m okay there. “Then we have an Apple TV app.” “Okay, Apple TV has a 4K version. So, it supports HEVC. All of the latest Apple TV devices have HEVC. That’s fine.” “Then we have smartphone apps, smart TV apps for Android TV and for the LG platform.”

Dror Gill: 29:15 Obviously, TV’s support 4K. So, I’m okay there. With delivering to mobile devices, all the high-end devices already support HEVC. He was making this estimate that around 50 to 60% of his viewers are using devices that are HEVC capable. Suddenly, he’s thinking, “Yeah, I can do that. I can go all in HEVC. I will continue, of course, to support H.264 for all of the devices that don’t support HEVC. But if I can save 50% of the bandwidth to 50 to 60% of my customers, that’s a very big savings.”

Mark Donnigan: 29:48 What’s interesting about this conversation, Dror, is first of all I’m pretty certain that the operator you’re talking with is different than the operator that I shared, found the exact same thing. This is a consistent theme, is that pretty much in developed parts of the world, it really is true that 50% or more of the users can today receive HEVC. This number is only growing. It’s not like it’s static It is just growing. Next year, I don’t know if that number will be 60% or 70%, but it’s going to be even bigger.

Mark Donnigan: 30:27 What’s fascinating is that, again, we’ve said earlier, that the consumer is getting just more aware of quality, and they’re getting more aware of when they’re being underserved. For operators who are serving to lowest common denominator, which is to say, AVC works across all my devices, and it’s true. AVC works on all the high-end devices equally well, but you’re under-serving a large and growing number of your users.

Mark Donnigan: 31:01 If your competitors are doing the same, then I guess you could say … well, “Who are they going to switch to?” But there are some fast-moving leaders in the space who are either planning or they’re shortly going to be offering better quality. They’re going to be extending HEVC into lower bit rates or lower resolutions, that is, and therefore lower bit rates, and the consumers are going to begin to see like, “Well, wait a second. This service over here that my friend has or we have another subscription in the household, how come the video looks better?” They just begin to migrate there. I think it’s really important when we have these sorts of conversations to connect to this idea that don’t underserve your consumer in an effort to be something to everybody.

Tim Siglin: 31:57 I would add two other quick things to that, Mark. One is, we’ve always had this conversation in the industry about the three-legged stool of speed, quality and bandwidth in terms of the encoding.

Mark Donnigan: 32:09 That’s right.

Tim Siglin: 32:09 Two of those are part of the consumer equation, which is quality and bandwidth. Then, oftentimes, we’ve had to make the decision between quality and bandwidth. If the argument is ostensibly that HEVC as it stands right now, had a couple years of optimization, can get us to about, let’s say, 40%. Let’s not even say 50%. For equivalent quality, it can get us to 40% bandwidth reduction. Why wouldn’t you switch over to something like that?

Tim Siglin: 32:39 Then the second part, and I have to put a plugin for what Eric Schumacher-Rasmussen and the Streaming Media team did at Streaming Media West by having Roger Pantos come and speak, Roger Pantos being of course the inventor of HLS, and I’m not a huge fan of HLS, just because of the latency issues. But he pointed out in his presentation, his tutorial around HLS that you can put two different codecs in a manifest file. There is absolutely no reason that an OTT provider could not provide both HEVC and AVC within the same manifest file and then allow the consumer device to choose.

Tim Siglin: 33:22 When Dror mentioned the company who has the OTT boxes that they give away, they could easily set a flag in those boxes to say, “If you’re presented with a manifest file that has AVC and HEVC, go with HEVC to lower the bandwidth, overall.” The beauty is it’s a technical issue at this point and it’s a technical implementation issue, not a ‘can we make it work?’ Because we know that it works based around the HLS.

Mark Donnigan: 33:54 This is excellent. Tim, let’s wrap this up, as I knew it would be. It has just been an awesome conversation. Thank you for sharing all your years of collective experience to give some insight into what’s happening in the industry. Let’s look at 2019. I know we’ve been talking a little bit about … you’ve made references to ATSC 3.0. Some of our listeners will be going to CES. Maybe there’s some things that they should be looking at or keeping their eyes opened for. What can you tell us about 2019?

Tim Siglin: 34:35 Here’s what I think 2019 is bringing. We have moved in the cloud computing space and you all are part of this conversation at Beamr. We’ve moved from having cloud-based solutions that were not at parity with on-premise solutions to actually in 2018 reaching parity between what you could do in an on-premise solution versus the cloud. Now, I think in 2019, what we’re going to start seeing is a number of features in cloud-based services, whether it’s machine learning, which the popular nomenclature is AI, but I really like machine learning as a much better descriptor, whether it’s machine learning, whether it’s real-time transcoding of live content, whether it’s the ability to simultaneously spit out AVC and HEVC like we’ve been talking about here that the cloud-based solutions will move beyond parity with the on-premise solutions.

Tim Siglin: 35:35 There always will be needs for the on-premise parts from a security standpoint in sort of the industries, but I don’t think that will inhibit cloud-based in 2019. If people are going to CES, one of the things to look at there, for instance, is a big leap in power consumption savings for mobile devices. I’m not necessarily talking about smartphones, because the research I’ve done says the moment you turn GPS on, you lose 25% of battery. Tablets have the potential to make a resurgence in a number of areas for consumers and I think we’ll see some advances in battery (capacity).

Tim Siglin: 36:19 Part of that goes to HEVC, which as we know is a much harder codec to decode. I think the consumer companies are being forced into thinking about power consumption as HEVC becomes more mainstream. That’s something I think people should pay attention to as well. Then, finally, HDR and surround sound solutions, especially object placement like Dolby Atmos and some of these others, will become much more mainstream as a way to sell flat panels and surround sound systems.

Tim Siglin: 36:56 We sort of languished in that space. 4K prices have dropped dramatically in the last two years, but we’re not yet ready for 8K. But I think we’ll see a trend toward fixing some of the audio problems. In the streaming space, to fix those audio problems, we need to be able to encode and encapsulate into sort of the standard surround sound model. Those are three areas that I would suggest people pay attention.

Mark Donnigan: 37:25 Well, thank you for joining us, Tim. It’s really great to have you on. We’ll definitely do this again. We want to thank you, the listener, for supporting the Video Insiders. Until the next episode. Happy encoding!

Announcer: 37:39 Thank you for listening to the Video Insiders Podcast, a production of Beamr Imaging Limited. To begin using Beamr’s codecs today, go to Beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.

Is AV1 more efficient than HEVC? Dror & Mark get into the middle of a 3 against 1 standoff over whether AV1 is actually more efficient than HEVC.

The following blog post first appeared on the Beamr blog at: https://blog.beamr.com/2018/11/23/codec-efficiency-is-in-the-eye-of-the-measurer-podcast/

When it comes to comparing video codecs, it’s easy to get caught up in the “codec war” mentality. If analyzing and purchasing codecs was as easy as comparing fuel economy in cars, it would undoubtedly take a lot of friction out of codec comparison, but the reality is that it’s not that simple.

In Episode 02, The Video Insiders go head-to-head comparing two of the leading codecs in a three against one standoff over whether AV1 is more efficient than HEVC.

So, which is more efficient?

Listen in to this week’s episode, “Codec Efficiency Is in the Eye of the Measurer,” to find out.

Want to join the conversation? Reach out to TheVideoInsiders@beamr.com.

TRANSCRIPTION (lightly edited to improve readability only)

Mark Donnigan: 00:41 Hi everyone I am Mark Donnigan and I want to welcome you to episode two of the Video Insiders.

Dror Gill: 00:48 And I am Dror Gill. Hi there.

Mark Donnigan: 00:50 In every episode of the Video Insiders we bring the latest inside information about what’s happening in the video technology industry from encoding, to packaging, to delivery, and playback, and even the business behind the video business. Every aspect of the video industry is covered in detail on the Video Insiders podcast.

Dror Gill: 01:11 Oh yeah, we usually do cover everything from pixels, to blocks, to microblocks, to frames, to sequences. We go all the way up and down the video delivery chain and highlight the most important things you should know before you send any video bits over the wire.

Mark Donnigan: 01:28 In our first episode we talked about a very hot topic which asked, “Hasn’t this kind of been worn out?” The whole HEVC, AV1 discussion. But I think it was very interesting. I sure enjoyed the talk. What about you Dror?

Dror Gill: 01:47 Yeah, yeah, yeah. I sure did. It was great talking about the two leading codecs. I don’t want to say the word, codec war.

Mark Donnigan: 01:58 No, no, we don’t believe in codec wars.

Dror Gill: 01:59 We believe in codec peace.

Mark Donnigan: 02:00 Yeah, that’s true. Why is it so complicated to compare video codecs? Why can’t it be as simple as fuel economy of cars, this one gets 20 miles per gallon and that one gets 30 and then I make a decision based on that.

Dror Gill: 02:15 I wish it was that simple with video codecs. In video compression you have so many parameters to consider. You have the encoding tools, tools are grouped into what’s called profiles and levels, or as AV1 calls them “experiments.”

Mark Donnigan: 02:31 Experiments, mm-hmm…

Dror Gill: 02:35 When you compare the codecs which profiles and levels do you use. What rate control method? Which specific parameters do you set for each codec? And each codec can have hundreds, and hundreds of parameters. Then there is the question of implementation. Which software implementation of the codec do you use? Some implementations are reference implementations that are used for research, and others are highly performance optimized commercial implementations. Which one do you select for the test? And then, which operating system, what hardware do you run on, and obviously what test content? Because encoding two people talking, or encoding an action scene for a movie, is completely different.

Dror Gill: 03:13 Finally, when you come to evaluate your video, what quality measure do you use? There’re various objective quality measures and some people use actual human viewers and they assesses subjective quality of the video. On that front also, there’re many possibilities that you need to choose from.

Mark Donnigan: 03:32 Yeah, so many questions and no wonder the answers are not so clear. I was quite surprised when I recently read three different technical articles published at IBC actually, effectively comparing AV1 versus HEVC and I can assume that each of the authors did their research independently. What was surprising was they came to the exact same conclusion, AV1 has the same compression efficiency as HEVC. This is surprising because some other studies and one in particular (I think we’ll talk about) out there says the contrary. So can you explain what this means exactly, Dror.

Dror Gill: 04:16 By saying that they have the same compression efficiency, this means that they can reach the same quality at the same bitrate or the other way round. You need the same bitrate to reach that same quality. If you need for example, two and a half megabits per second to encode an HD video file using HEVC at a certain quality, then with AV1 you would need roughly the same bitrate to reach that same quality and this means that AV1 and HEVC provide the same compression level. In other words, this means that AV1 does not have any technical advantage over HEVC because it has the same compression efficiency. Of course that’s if we put aside all the loyalty issues but we discussed that last time. Right?

Mark Donnigan: 04:56 That’s right. The guys who wrote the three papers that I’m referencing are really top experts in the field. It’s not seminar work done by a student, not to downplay those papers, but the point is these are professionals. One was written by the BBC in cooperation with the Multimedia and Vision Group at the Queen Mary University of London. I think nobody is going to say that the BBC doesn’t know a thing or two about video. The second was written by Ateme, and the third by Harmonic, leading vendors.

Mark Donnigan: 05:29 I actually pulled out a couple of phrases from each that I’d like to quote. First the BBC and Queen Mary University, here is a conclusion that they wrote, “The results obtained show in general a similar performance between AV1 and the reference HEVC both objectively and subjectively.” Which is interesting because they did take the time to both do the visual assessment as well as use a quality measure.

Mark Donnigan: 06:01 Ateme said, “Results demonstrate AV1 to have equivalent performance to HEVC in terms of both objective and subjective video quality test results.”

Dror Gill: 06:10 Yeah, very similar.

Mark Donnigan: 06:16 And then here is what Harmonic said, “The findings are that AV1 is not more advantageous today than HEVC on the compression side and much more complex to encode than HEVC.” What do you make of this?

Dror Gill: 06:32 I don’t know. It sounds pretty bad to me, even two of those papers also analyzed subjective quality so they used actual human viewers to check out the quality. But Mark what if I told you that researchers from the University of Klagenfurt in Austria together with Bitmovin published a paper which showed completely different results. What would you say about that?

Mark Donnigan: 06:57 Tell me more.

Dror Gill: 06:58 Last month in Athens I was the ICIP conference that’s the IEEE International Conference on Image Compression and Image Processing. There was this paper presented by this University in Austria with Bitmovin and their conclusion was, let me quote, “When using weighted PSNR, AV1 performs consistently better for bit rate compared to AVC, HEVC, and VP9.” So they claim AV1 is better than three codecs but specifically it’s better than HEVC. And then they have a table in their article that compares AV1 to HEVC for six different video clips. The table shows that with AV1 you get up to 25% lower bitrate at the same quality than HEVC.

Dror Gill: 07:43 I was sitting there in Athens last month when they presented this and I was shocked.

Mark Donnigan: 07:50 What are the chances that three independent papers are wrong and only this paper got it right? And by the way, the point here is not three against one because presumably there’re some other papers. I’m guessing other research floating around that might side with Bitmovin. The point is that three companies who no one is going to say that any of them are not experts and not highly qualified to do a video assessment, came up with such a different result. Tell us what you think is going on here?

Dror Gill: 08:28 I was thinking the same thing. How can that be. During the presentation I asked one of the authors who presented the paper a few questions and it turned out that they made some very questionable decisions in all of that sea of possibility that I talked about before. Decisions related to coding tools, codec parameters, and quality measures.

Dror Gill: 08:51 First of all, in this paper they didn’t show any results of subjective viewing. Only the objective metrics. Now we all know that you should always your eyes, right?

Mark Donnigan: 09:03 That’s right.

Dror Gill: 09:04 Objective metrics, nice numbers, but obviously you need to view the video because that’s how the actual viewers are going to assess the (video) quality. The second thing is that they only used the single objective metric and this was PSNR. PSNR, it stands for peak signal-to-noise ratio and basically this measure is a weighted average of the difference in peaks between pixel values of the two images.

Dror Gill: 09:30 Now, we’re Video Insiders, but even if you’re not an insider you know that PSNR is not a very good quality measure because it does not correlate very well with human vision. This is the measure that they choose to look at but what was most surprising is that there is a flag in the HEVC open source encoder which they used that if chosen, the result is improved PNSR. What it does, it turns off some psycho-visual optimizations which make the video look better but reduce the PSNR, and that’s turned on by default. So you would expect that they’re measuring PSNR they would turn that flag on so you would get higher PSNR. Well, they didn’t. They didn’t turn the flag on!

Mark Donnigan: 10:13 Amazing.

Dror Gill: 10:17 Finally, even then AV1 is much slower than HEVC, and they also reported in this data that it was much, much slower than HEVC but still they did not use the slowest encoding standing of HEVC, which would provide the best quality. There’s always a trade off between performance and quality. The more tools you employ the better quality you can squeeze out of the video, of course that takes you more CPU cycles but they used for HEVC, the third slowest setting which means this is the third best quality you can get with that codec and not the very best quality. When you handicap an HEVC encoder in this way, it’s not surprising that you get such poor results.

Dror Gill: 11:02 I think based on all these points everybody can understand why the results of this comparison were quite different than all of the other comparison that were published a month earlier at IBC (by Ateme, BBC, Harmonic).

Mark Donnigan: 11:13 It’s interesting.

Mark Donnigan: 11:14 Another critical topic that we have to cover is performance. If you measure the CPU performance on encoding time of AV1, I believe that it’s pretty universally understood that you are going to find it currently is a hundred times slower than HEVC. Is that correct?

Dror Gill: 11:32 Yeah, that’s right. Typically, you measure the performance of an encoder and FPS which is frames per second. For HEVC it’s common to measure an FPM which is frames per minute.

Mark Donnigan: 11:42 Frames per minute, (more like) frames per hour, FPH.

Dror Gill: 11:45 A year and a half ago or a year ago when there were very initial implementation, it was really FPD or FPH, Frames per hour or per day and you really needed to have a lot of patience, but now after they’ve done some work it’s only a hundred times slower than HEVC.

Mark Donnigan: 12:02 Yeah, that’s pretty good. They’re getting there. But some people say that the open source implementation of AV1 I believe it’s AOM ENC.

Dror Gill: 12:11 Yeah, AOM ENC.

Mark Donnigan: 12:16 ENC exactly has not been optimized for performance at all. One thing I like about speed is either your encoder produces X number of frames per second or per minute, or it doesn’t. It’s really simple. Here is my next question for you. Proponents of AV1 are saying, “well it’s true it’s slow but it hasn’t been optimized, the open source implementation,” which is to imply that there’s a lot of room (for improvement) and that we’re just getting started, “don’t worry we’ll close the gap.” But if you look at the code, and by the way I may be a marketing guy but my formal education is computer science.

Mark Donnigan: 13:03 You can see it already includes performance optimizations. I mean eptimizations like MMX, SSE, there’s AVX instructions, there’s CPU optimization, there’s multithreading. It seems like they’re already trying to make this thing go faster. So how are they going to close this a hundred X (time) gap?

Dror Gill: 13:22 I don’t think they can. I mean a hundred X, that’s a lot and you know even the AV1 guys they even admit that they won’t be able to close the gap. I talked to a few senior people who’re involved in the Alliance for Open Media and even they told me that they expect AV1 to five to 10 times more complex than HEVC at the end of the road. In two to three years after all optimization are done, it’s still going to be more complex than HEVC.

Dror Gill: 13:55 Now, if you ask me why it’s so complex I’ll tell you my opinion. Okay, this is my personal opinion. I think it’s because they invested a lot of effort in side stepping the patents (HEVC).

Mark Donnigan: 14:07 Good point. I agree.

Dror Gill: 14:07 They need to get that compression efficiency which is the same as HEVC but they need to use algorithms that are not patented. They have methods that use much more CPU resources than the original patent algorithms to reach the same results. You can call it kind of brute force implementation of the same thing to avoid the patent issue. That’s my personal opinion, but the end result I think is clear, it’s going to be five to 10 times slower than HEVC. It has the same compression efficiency so I think it’s quite questionable. This whole notion of using AV1 to get better results.

Mark Donnigan: 14:45 Absolutely. If you can encode let’s say on a single computer with HEVC a full ABR stack, this is what people want to do. But here we’re talking speeds that are so slow let’s just try and do (encode) one stream. Literally what you’re saying is you’ll need five to 10 computers to do the same encode with AV1. I mean, that’s just not viable. It doesn’t make sense to me.

Dror Gill: 15:14 Yeah, why would you invest so much encoding into getting the same results. If you look at another aspect of this, let’s talk about hardware encode. Companies that have large data centers, companies that are encoding vast amount of video content are not looking into moving from the traditional software encoding and CPUs and GPUs, to dedicated hardware. We’re hearing talks about FPGAs even ASICs … by the way this is a very interesting trend in itself that we’ll probably cover in one of the next episodes. But in the context of AV1, imagine a chip that is five to 10 times larger than an HEVC chip and which is the same complexity efficiency. The question I ask again is why? Why would anybody design such a chip, and why would anybody use it when HEVC is available today? It’s much easier to encode, royalty issues have been practically solved so you know?

Mark Donnigan: 16:06 Yeah, it’s a big mystery for sure. One thing I can say is the Alliance for Open Media has done a great service to HEVC by pushing the patent holders to finalize their licensing terms … and ultimately make them much more rational shall we say?

Dror Gill: 16:23 Yeah.

Mark Donnigan: 16:25 Let me say that as we’re an HEVC vendor and speaking on behalf of others (in the industry), we’re forever thankful to the Alliance for Open Media.

Dror Gill: 16:36 Definitely, without the push from AOM and the development of AV1 we would be stuck with HEVC royalty issue until this day.

Mark Donnigan: 16:44 That was not a pretty situation a few years back, wow!

Dror Gill: 16:48 No, no, but as we said in the last episode we have a “happy ending” now. (reference to episode 1)

Mark Donnigan: 16:52 That’s right.

Dror Gill: 16:52 Billions of devices support HEVC and royalty issues are pretty much solved, so that’s great. I think we’ve covered HEVC and AV1 pretty thoroughly in two episodes but what about the other codecs? There’s VP9, you could call that the predecessor of AV1, and then there’s VVC, which is the successor of HEVC. It’s the next codec developed by MPEG. Okay, VP9 and VVC I guess we have a topic for our next episode, right?

Mark Donnigan: 17:21 It’s going to be awesome.

Narrator: 17:23 Thank you for listening to the Video Insider podcast a production of Beamr limited. To begin using Beamr codecs today go to beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.

Join the conversation by jumping into The Video Insiders LinkedIn Group.

For more podcast episodes, visit our Show Guide.

For more information on Beamr, visit us online.

Read the accompanying blog post here.

If you would like to be a guest on the show, send an email to thevideoinsiders@beamr.com