iSIZE, a market innovator in deep learning for video delivery, has developed BitClear, an AI-based video processing technology that removes compression artifacts (like blurring and blocking artifacts) from user-generated (or heavily-compressed) content. Video content that has been through multiple transcoding iterations can be revived to the maximum possible quality without affecting the artistic intent of the original creators. The process can also allow for video upscaling, all with as little as 5ms processing latency on GPUs or high-performance CPUs.
“The huge traffic in shared video content today passes through several platforms and users before potentially reaching a large audience or big influencers,” explained Sergio Grce, CEO and founder of iSIZE. “Quite often, UGC that is viewed by millions of people over multiple social media platforms has been re-shared and re-uploaded numerous times. This process tends to make it very degraded or even unwatchable due to multiple transcoding iterations.
“For social media or UGC distribution and streaming companies, it is imperative that video content is presented in as high quality as possible to retain audiences,” he continued. “BitClear is designed to achieve the maximum quality improvement by alleviating the effect of compression artifacts. Its AI nature and implementation efficiency means it can be deployed at scale without the need for human inspection or tuning.”
Other applications of BitClear include real-time de-noising before transcoding and broadcasting of live game streams, live sports, entertainment, and virtual reality feeds.
Through bespoke neural network designs that optimally learn to disentangle the noise from the data manifolds, BitClear is able to remove compression noise and retain or recover the original content features. While it learns the noise signatures of the various encoding standards, BitClear does not need to know the history of the specific asset: it can process any highly-compressed content and produced a higher-quality output that improves the value of the asset.
Unlike previous noise removal technologies that aim towards specific noise patterns like film grain or blur, BitClear specifically targets compression artifacts of typical MPEG or AOMedia encoders, with particular focus on highly-compressed or highly-distorted content.
On an Intel Xeon CPU, the fastest BitClear models achieve 30 frames per second for 1080p content. Its scalable neural network architecture means it can scale up to high volumes of content and is ideal for cloud as well as on-premise deployment on CPUs, GPUs or custom hardware that supports neural network inference.
“BitClear is currently in evaluation trials with major UGC distribution platforms,” said Grce. “While the current focus is on removal of compression artifacts from heavily-compressed user generated content, our unique AI-powered technology can also be trained for other denoising tasks or to optimize delivery where limited by infrastructures, for example in mobile streaming and IoT security.”
iSIZE, a specialist in deep learning for video delivery, today announces that it has joined the AWS Partner Network (APN), the global community of businesses using Amazon Web Services (AWS) to build solutions and services for customers.
In addition to joining the APN, iSIZE has completed an AWS Foundational Technical Review (FTR) of its award-winning, patented BitSave deep psychovisual pre-processing technology. BitSave pre-processes the input video prior to encoding and removes imperceptible information that is costly to encode by all existing video encoders. It delivers greater efficiency over the non-preprocessed content encoding, i.e., 15%-25% bitrate and energy savings over the utilized encoder, verified by standard quality metrics (VMAF, SSIM, etc.) that are widely used in the industry. FTR enables AWS Partners to identify and remediate risks in products or solutions. The FTR is led by an AWS Partner Solutions Architect (PSA) who reviews AWS Partner products and solutions against a specific set of requirements based on the Security, Reliability, and Operational Excellence pillars of the AWS Well-Architected Framework.
iSIZE CEO Sergio Grce commented, “We are delighted to join the AWS Partner Network. By being a part of the APN, we can enable our customers to maximise existing technology investments, benefit from greater efficiencies when processing video streams, and provide better video streaming experiences to millions of users worldwide.”
Using iSIZE’s technology prior to conventional encoding makes encoding significantly more efficient in terms of bitrate saving at the same or improved visual quality. An encoder agnostic solution, BitSave also allows for the encoder to be used with simpler encoding recipes, thereby making it faster, saving datacenter cost and significantly improving the sustainability of video streaming. It requires no changes to the encoding, delivery or decoding devices.
iSIZE today announces that it has won the 2021 Digital TV Europe (DTVE) VideoTech Innovation Award for Sustainability with its BitSave pre-processor for video.
With global internet traffic set to reach 4.8 zetabytes a year in 2022, and 80% of this resulting from video, there is a significant environmental cost. Today the internet is using an estimated 10% of the world’s total energy consumption, putting its carbon footprint at the same level as air travel. Anything that can reduce the traffic on the internet means a reduction in that environmental impact. The challenge is working with an infrastructure for distributing online and mobile video that is already well established and largely unchangeable. Upgrading codecs takes time and deployment risks are high. Enter BitSave.
iSIZE's BitSave preprocesses the input video prior to encoding and removes imperceptible information that is costly to encode by all existing video encoders. By being encoder agnostic, BitSave brings benefits to all video coding standards, like AVC/H.264, HEVC/H.265, VVC/H.266 or VP9 and AV1. No changes are required in the encoding, stream packaging, streaming and decoding.
Using iSIZE’s deep neural network solution, developed specifically for media content, means valuable bandwidth reduction without compromising the quality. To create BitSave, the iSIZE team undertook extensive scientific research on visual perception and visual quality scoring to deliver a video preprocessing solution that preserves all visually salient characteristics of each input frame, while attenuating details that are imperceptible by viewers and incur significant cost when encoding with standard encoders.
Sergio Grce, CEO at iSIZE commented, “We are delighted to have been recognised for our contribution to making video streaming more sustainable. Instead of the traditional approach of using simple signal-to-noise analysis or broad-brush bitrate reduction, we have deployed our rich artificial intelligence and machine learning to process individual video streams dynamically. While it is a highly sophisticated solution, BitSave runs on standard workstation hardware, making it simple and easy to implement. The consumer benefits from a more reliable service even in marginal conditions, because of the reduced bandwidth.”
iSIZE has been selected to present at this year’s SMPTE Annual Technical Conference (SMPTE 2021 ATC) starting Nov. 9 and running select days through Nov. 18. The conference will be a virtual event with technical paper presentations, SMPTE Storytellers, roundtables, and networking events.
Yiannis Andreopoulos of iSIZE will be presenting a paper on the first day entitled “Toward Generalized Psychovisual Preprocessing For Video Encoding”.
“Deep perceptual preprocessing has recently emerged as a new way to enable further bitrate savings across several generations of video encoders without breaking standards or requiring any changes in client devices”, explained Andreopoulos. He went on, “In this paper, we lay the foundations toward a generalized psychovisual preprocessing framework for video encoding and describe one of its promising instantiations that is practically deployable for video-on-demand, live, gaming and user-generated content."
The paper will demonstrate the results using state-of-the-art AVC, HEVC and VVC encoders show that average bitrate (BD-rate) gains of 11% to 17% are obtained over three state-of-the-art reference-based quality metrics (Netflix VMAF, SSIM and Apple AVQT), as well as the recently-proposed non-reference ITU-T p.1204 metric. The runtime complexity of the proposed framework on CPU is shown to be equivalent to a single x264 medium-preset encoding. On GPU hardware, the iSIZE approach achieves 260fps for 1080p video (below 4ms/frame), thereby enabling its use in very-low latency live video or game streaming applications.
To watch this virtually, please register here.
The Alliance for Open Media (AOMedia) today announced that iSIZE, a company with a codec-independent AI technology that reduces bandwidth use and improves the video experience, has joined the organization at the Promoter level. As a member of the Alliance, iSIZE will collaborate with AOMedia members, the leading internet and media technology companies, to advance open standards for media compression and delivery over the web while promoting hardware video encoding adoption.
iSIZE specializes in deep learning for intelligent and sustainable video delivery. Its proprietary deep perceptual optimizer uses AI trained to "see with the human eye" in order to optimize video quality, and it offers significant bitrate savings for all video encoding standards, including AV1.
"We are very pleased to join AOMedia, where we can share our experience with AI-based solutions for high-quality video streaming," said Sergio Grce, CEO at iSIZE. "Like AOMedia, we are passionate about building high-impact video solutions that leave a mark on the world, and we look forward to working with our fellow members to deliver superb video streaming experiences."
As a result of AV1's improved data compression over existing standards, fewer bits need to be streamed to reach high visual quality levels and improved user experience. iSIZE's BitSave AI-based preprocessor solutions can be used in tandem with any VP9 or AV1 encoder to improve the quality of VP9 or AV1 video on client devices without requiring any change on the streaming or decoding devices.
"It's great to welcome iSIZE to AOMedia," said Matt Frost, AOMedia Chair and Director at Google. "iSIZE's expertise in bitrate and quality gains in video compression will benefit the entire video streaming ecosystem. We look forward to collaborating with iSIZE on our goal to expand hardware adoption of the AV1 standard in hardware and increase openness and interoperability of internet video."
iSIZE, the AI-tech company, that specializes in deep learning for intelligent and sustainable video delivery, is exhibiting as part of the Connected Media|IP at NAB 2021 and will show the latest version of BitSave, its fully artificial intelligence (AI)-powered technology. This first-of-a-kind solution leverages deep perceptual optimization in order to optimize video quality, and it offers significant bitrate savings for all video encoding standards.
ISIZE has also been selected to present a paper – entitled Perceptually-Optimized AI-Based Video Delivery Over Existing Standards – on Sunday, October 10 at 15:20, presented by the company’s CTO and Co-Founder, Yiannis Andreopoulos. Additionally, Andreopoulos is participating in a panel session entitled Meeting the Demand for a Content Everywhere Future.
The latest updates to BitSave mean it is now capable of achieving even greater bitrate savings, along with a leap forward in video quality and optimization. iSIZE is also demonstrating a new AI-based video codec de-noising technology and a degenerative video compression innovation at NAB 2021. A server-side preprocessing enhancement, BitSave gives customers an elegant way to integrate AI with existing systems and encoders to achieve the savings without moving to new codec standards.
iSIZE CEO Sergio Grce said, “The accelerating demand for higher quality video streaming and the need to deliver their video content as cost efficiently as possible are challenges that everyone who streams media now faces. We know that the amount of video content is only going to increase and that the demand for higher resolution content is on an upward trend – Cisco forecasts that 56.8 percent of the global IP video traffic will be HD and 22.3 percent will be Ultra HD by 2022. Studies also show that the growing carbon footprint of video streaming has overtaken that of the airline industry. BitSave addresses all three of these major challenges in a cost-efficient way without compromising the end viewer experience.”
iSIZE’s patent-pending technology is fully codec independent, increasing the efficiency and performance of all latest codec standards including AVC/H.264, HEVC/H.265, and VP9. This ensures seamless integration with existing media workflows, without breaking any standards or requiring any changes in the streaming process or the client devices.
By iSIZE staff | iSIZE technologies | Published 29th April 2020
Enhanced video streaming start-up iSIZE Technologies has today announced the appointment of Paul Massara, Ex-CEO of Npower, and Maria Ingold, Ex-CTO of Disney-Sony joint venture FilmFlex Movies, to its Executive Board.
Massara brings over two decades of experience in the energy sector to the role, after various executive positions at Centrica plc, Northstar Solar and Habitat Energy, before becoming CEO at energy giant Npower. With a similarly impressive track record, Ingold joins the Board with extensive technical experience, having helped innovate the beginnings of audio and video on PCs at IBM and early 3D PC computer gaming at Ocean, before becoming a Senior Technical Executive in streaming, including a 5-year tenure as CTO at Disney-Sony joint venture FilmFlex Movies, which produced one of the most successful on demand film services in Europe.
Commenting on his appointment, Massara said: “I am very excited to join iSIZE Technologies, a start-up which is shaking up the streaming industry. Having dedicated much of my career to furthering the environmental cause, I am very much looking forwards to taking iSIZE low-carbon, sustainable technology solutions to new heights in 2020.”
Speaking about the role, Ingold commented: “I’m thrilled to be part of iSIZE at such a relevant time. With nearly 30 years of technical expertise in the entertainment and media industry, I am well-placed to see the potential iSIZE has to revolutionise the sector and am delighted to be part of its future.”
Sergio Grce, CEO at iSIZE Technologies, added: “We are very pleased to have both Paul and Maria on board here at iSIZE. With years of commercial and technical experience under their belts, these appointments demonstrate our accelerated growth trajectory as we continue to expand our corporate horizons in 2020.
With Paul’s impressive business acumen and Maria’s technical credentials on side, we are confident that iSIZE will continue to flourish as we look to make our mark in the streaming sector in 2020.”
iSIZE Technologies, a London-based provider of intelligent video coding and delivery technology, launched its pioneering AI-powered encoding platform in 2016. Last year, the innovative start-up was awarded a special merit prize at The Digital Media World awards for its proprietary BitSave software, a cloud transcoding technology which, by compressing vast quantities of data, significantly reduces the energy input required to stream videos.
By Maria Ingold | iSize Technologies | Published 21st April 2020
I grew up off-grid in a cabin in the New Mexico mountains. That was isolation. By contrast, isolation in the time of coronavirus is incredibly connected. While working, socialising and relaxing from home have impacted that connectivity, new patterns are emerging as well as opportunities for the future.
What is the scope of the impact?
Akamai, a content delivery network (CDN), saw global internet traffic increase by 30% in March, an entire year’s growth, and without live sports streaming. Comcast saw a 32% increase in peak USA traffic over March, with plateaus in early lockdown markets.
Even before COVID-19 video was 60% of downstream internet traffic. When Conviva analysed three weeks in mid-March they discovered that video streaming viewing hours jumped more than 20% globally in that last week, up 27% in the USA. By the end of March, Comcast saw a 38% USA increase.
Although Internet service providers (ISPs) and CDNs are engineered to deal with peak changes, when usage spiked, the European Commissioner asked streamers to switch to Standard Definition (SD) when High Definition (HD) wasn’t necessary. The European Broadcasting Union (EBU) then issued recommendations for adapting streaming quality during times of crisis.
Assuaging one concern, Conviva discovered that daytime viewing jumped nearly 40%, spreading peak load, but that still leaves lots of bits flowing across the internet.
Netflix and Google’s YouTube agreed to reduce bitrates in Europe for 30 days, with Netflix dropping by 25% and YouTube moving to SD as a default globally. Both were crucial, because while Netflix usually has the largest percentage of video traffic, YouTube is currently generating almost twice the traffic of Netflix. Amazon Prime Video, Apple TV+ and Walt Disney’s Disney+ soon followed.
Consumers were concerned. They were paying for HD but would get SD. Netflix explained that customers would still get the SD, HD and Ultra-High Definition (UHD) resolutions they paid for, just no longer the highest quality from a “bitrate ladder” of low to high bitrates and resolutions.
What are the long-term opportunities?
Netflix’s total energy consumption for 2019–451,000 megawatt-hours — could power 40,000 average American homes for a year, at an 84% increase over 2018, compared to 20% user growth.
Netflix has 167m subscribers. Disney+ has 50m, with 226m subs predicted by 2024. Reducing bits creates a more sustainable energy-consumption to user-growth ratio and helps companies meet their environmental impact objectives.
During the 30-days of COVID-19-inspired bitrate reduction, streamers will have saved money by reducing storage, distribution and energy costs. If one million people watch one hour per day, at 1 GB of data per hour (somewhere between SD and 720p HD), and it costs .0025 USD to stream 1 GB to one person, that’s nearly $1 million per year ($912,500). YouTubers watch one billion hours per day. That’s nearly $1 billion per year. A 25% savings is $228 million.
While these short-term actions enabled quick bitrate reductions and increased margins, they don’t preserve quality. Consumers won’t tolerate that indefinitely.
How to cut costs and maintain customer satisfaction
A codec encodes (usually in hardware) the moving image source and decodes (usually in software) on a device to display it. It reduces bitrate as much as possible while attempting to maintain fidelity. Codecs range from older, widely supported but higher bitrate like MPEG-4 AVC (H.264) to newer, less supported, time and power-hungry but lower bitrate ones.
Per-title encoding is a bitrate reduction tactic pioneered in 2015 by Netflix. To measure fidelity, Netflix used quality metric PSNR (Peak Signal-To-Noise Ratio), but it doesn’t always measure how it looks to a person. Neither does SSIM (structural similarity), designed to improve on PSNR. So, Netflix co-created VMAF (Video Multi-Method Assessment Fusion), a perceptual quality metric.
Machine learning (ML) can reverse engineer perceptual metrics to make encoding more effective. When this precedes encoding — precoding — it works with any codec, encoder and decoder. There are trade-offs between the sharpness of VMAF, which can look artificial, the naturalness of PSNR and SSIM, and the blurriness and lack of fidelity caused by reducing bitrate.
I advise iSIZE, a machine learning precoder that claims 20%-40% bitrate savings (up to 60%) without changing the resolution and typically improving VMAF. Latency is one frame. I asked expert reviewer Jan Ozer to independently test iSIZE’s BitSave product. He tested using the MPEG AVC (H.264) codec.
Jan confirmed that “BitSave is a legitimate processing technology and not a [VMAF] hack”. Ultimately, “[a]fter many hours of testing, [Jan] found that BitSave’s technology is valid and valuable” though he recommends subjective testing. I agree and recommend testing various bitrate savings and metric balances. Regardless of the solution you choose, remember to balance long-term sustainability and cost-cutting with perceived customer experience.
By Adrian Pennington | IBC 365 | Published 27th April 2020
The traditional means of optimising video streaming workflows have run their course. Future advances will be made in software automated by AI.
Online video providers have never been under so much pressure. Excess demand has caused Netflix, YouTube and Disney+ to tune down their bitrates and ease bandwidth consumption for everyone, in the process deliberately compromising the ultimate quality of their service.
Even once the crisis has subsided operators will have to equate scaling growth with the cost of technical investment and bandwidth efficiency. Even in a world with universal 5G, bandwidth is not a finite resource.
For example, how can an esports streaming operator grow from 100,000 to a million simultaneous live channels and simultaneously transition to UHD?
“Companies with planet scale steaming services like YouTube and Netflix have started to talk about hitting the tech walls,” says Sergio Grce, CEO at codec developer iSize Technologies. “Their content is generating millions and millions of views but they cannot adopt a new codec or build new data centres fast enough to cope with such an increase in streaming demand.”
By iSIZE | edie.net | Published 19th June 2020
Episode 87 of the edie’s Sustainable Business Covered podcast focuses on the energy impact of streaming and digitization and features interviews with iSIZE CTO and UCL professor, Yiannis Andreopoulos along with former NPower Chief Executive, Paul Massara, who was recently appointed to iSIZE’s executive board.
Andreopoulos and Massara discuss the need to balance the environmental benefits of switching meetings to online platforms with the energy intensity of data centres and the online services we access.