Welcome!

WebRTC Summit Authors: Yeshim Deniz, Carmen Gonzalez, Pat Romanski, Elizabeth White, Liz McMillan

Related Topics: @DevOpsSummit, Linux Containers, Containers Expo Blog

@DevOpsSummit: Blog Feed Post

The Speed Index | @DevOpsSummit @Catchpoint #DevOps #WebPerf #DX #AI

There’s a lot of buzz around the speed index metric, but what exactly is this metric and how is it calculated?

Understanding Speed Index
By Moiz Khan

There’s a lot of buzz around the speed index metric, but what exactly is this metric and how is it calculated? Even more so, why do we need a new metric when there are so many already on the plate? This article gives you detailed answers to these questions.

What is speed index?
User expectations have changed drastically, webpages are expected to load faster with all elements rendering as soon as possible; user frustration increases if the visual content of the page is slow to load.

There are several metrics used to measure user experience, but do these give us a clear enough picture? Not really. For example, render start indicates when the page started painting, but this could mean it loaded a white background, a random object, or just a dot on the page; the user, however, may still be looking at a blank page. The document complete metric, which flags the onLoad event, may not be important from an end user’s perspective as it considers below-the-fold requests as well; there could be instances where the Document complete is triggered even when the page is blank.

Speed Index is an abstract score that was introduced to overcome the limitations of the current set of metrics. It measures user experience and unlike render start or document complete, it is not a timing metric, therefore the smaller the score, the better it is. Speed index takes into account the visual completeness of the page above-the-fold. It does a much better job of measuring the perceived performance of a page, but does this mean other metrics are irrelevant? This is debatable. Instead, the choice of a metric depends on the scenario and the nature of the website being analyzed.

How relevant is speed index?
Speed index will let you know if the visible content is loading quickly or not. The lower the score, the better is the user experience, and vice versa. The speed index score indicates whether you need to optimize your page or not. To maintain a good score, there are two critical aspects that you should look at: optimizing content efficiency and optimizing the critical rendering path.

Web applications continue to grow in terms of functionality, scope, and usability; hundreds of resources make up these applications and fetching each resource adds to the page load time. Maintaining performance is possible only when the resources and content that make up the page are optimized. Speed index plays a critical role in monitoring the performance of visible content.

How is it calculated?
Speed index lets you measure how quickly a page’s content is visible. However, it can be a little tricky to understand what it means and what factors contribute to the score.

Speed index uses film strips to calculate the score, each frame is scored for visual completeness above the fold – the score is 0% for a blank screen and 100% for a visually complete screen. The score for each frame is calculated using this formula:

interval time*(1 – visual complete %/100)

The process is repeated for every frame and the cumulative of all the scores gives you the speed index of that page.

Let’s take an example:

Adding all of the scores gives the speed index score:

500 + 450 + 350 + 200 + 50 = 1550

Here is another example:

Let’s add up the score:

500 + 500 + 475 + 400 + 350 = 2225

From a performance point of view, the lower the score the better it is. There is no defined benchmark for speed index, however, a score of <1000 is considered good. User experience is better if above-the-fold content is displayed faster, and this is what speed index determines.

Measuring speed index using Catchpoint
To check speed index, you will need to enable the “Capture Filmstrip” option under Advanced Settings of a test. It is displayed along with other metrics in the charts, and you can also view it in the waterfall graphs. Here are a few examples that illustrate how pages that load the visual content faster have a better speed index score.

Example 1:

In the above example, the page has a good speed index score of 1260. If you look at the film strip, the interval is of 500ms, and from the second frame the visual content is visible, which is about 1 second. Frame three has almost all the content loaded. From a user’s point of view, we have engaged them from the very first second due to the fact that the page is optimized.

Example 2:

In the above image, the visual is visible in the third frame. The first two frames are blank, which is why the speed index score of 1741 is higher than the first example. The user views content at 1.6 seconds, unlike the first example where the user viewed the content within a second.

Example 3:

Based on the final example above, we are able to understand what is happening. The first two frames are blank and in the third frame visual content starts to load. However, the completeness is less compared to the other two images in the same frame. The visual content continues to load in the fourth frame, but it’s not complete until the fifth frame. This means that the user needs to wait for 1.6 seconds to see something on the page, and another 2.8 seconds to view the complete visual content.

In all of the examples, the render start is triggered before the user views anything on the page. In the second example, the render start happens at 1195ms and the visible content starts to load at 1604ms. In the third example, the render start happens at 1296ms and the visible content starts to load at 1597ms. This is the uniqueness of speed index. With a highly unoptimized website, the difference could be extremely high, with render start happening quickly but the visible content loads quite late. You need these insights to optimize your websites.

Limitations of Speed Index
When running transaction tests for dynamic sites or single page applications that use ajax, the score is always good in the steps that follow once the page is opened. This is because the page does not refresh after it is loaded. Pages featuring carousels that rotate automatically may be penalized because they continue to change even after the page load is completed.

Summary
Speed index indicates the visual completeness of the page; however, it does not indicate if the content was critical or non-critical for the user. As such, it should not be a replacement for other metrics and should be considered a rather useful and much-needed addition to the list of metrics. You will still need to look at render start and document complete to understand the overall performance. Speed index gives you a different perspective of the user experience that your site delivers.

The post Understanding Speed Index appeared first on Catchpoint's Blog - Web Performance Monitoring.

More Stories By Mehdi Daoudi

Catchpoint radically transforms the way businesses manage, monitor, and test the performance of online applications. Truly understand and improve user experience with clear visibility into complex, distributed online systems.

Founded in 2008 by four DoubleClick / Google executives with a passion for speed, reliability and overall better online experiences, Catchpoint has now become the most innovative provider of web performance testing and monitoring solutions. We are a team with expertise in designing, building, operating, scaling and monitoring highly transactional Internet services used by thousands of companies and impacting the experience of millions of users. Catchpoint is funded by top-tier venture capital firm, Battery Ventures, which has invested in category leaders such as Akamai, Omniture (Adobe Systems), Optimizely, Tealium, BazaarVoice, Marketo and many more.

@WebRTCSummit Stories
SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 21st Int\ernational Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their online business and let Enzu manage their IT hosting infrastructure.
SYS-CON Events announced today that GrapeUp, the leading provider of rapid product development at the speed of business, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market across the USA and Europe, we work with a variety of customers from emerging startups to Fortune 1000 companies.
SYS-CON Events announced today that Ayehu will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara California. Ayehu provides IT Process Automation & Orchestration solutions for IT and Security professionals to identify and resolve critical incidents and enable rapid containment, eradication, and recovery from cyber security breaches. Ayehu provides customers greater control over IT infrastructure through automation. Ayehu solutions have been deployed by major enterprises worldwide, and currently, support thousands of IT processes across the globe. The company has offices in New York, California, and Israel.
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex software systems for startups and enterprises. Since 2009 it has grown from a small group of passionate engineers and business managers to a full-scale mobile software company with over 200 developers, designers, quality assurance engineers, project managers in house, specializing in the world-class mobile and web development.
In his keynote at @ThingsExpo, Chris Matthieu, Director of IoT Engineering at Citrix and co-founder and CTO of Octoblu, focused on building an IoT platform and company. He provided a behind-the-scenes look at Octoblu’s platform, business, and pivots along the way (including the Citrix acquisition of Octoblu).
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, which can process our conversational commands and orchestrate the outcomes we request across our personal and professional realm of connected devices.
SYS-CON Events announced today that Outscale will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Outscale's technology makes an automated and adaptable Cloud available to businesses, supporting them in the most complex IT projects while controlling their operational aspects. You boost your IT infrastructure's reactivity, with request responses that only take a few seconds.
SYS-CON Events announced today that Progress, a global leader in application development, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Enterprises today are rapidly adopting the cloud, while continuing to retain business-critical/sensitive data inside the firewall. This is creating two separate data silos – one inside the firewall and the other outside the firewall. Cloud ISVs often get requests to connect these silos using technologies such as VPN; however, these tend to be difficult to manage and are not engineered for accessing business data from the cloud.
SYS-CON Events announced today that Progress, a global leader in application development, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Enterprises today are rapidly adopting the cloud, while continuing to retain business-critical/sensitive data inside the firewall. This is creating two separate data silos – one inside the firewall and the other outside the firewall. Cloud ISVs often get requests to connect these silos using technologies such as VPN; however, these tend to be difficult to manage and are not engineered for accessing business data from the cloud.
SYS-CON Events announced today that Outscale, a global pure play Infrastructure as a Service provider and strategic partner of Dassault Systèmes, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 2010, Outscale simplifies infrastructure complexities and boosts the business agility of its customers. Outscale delivers a secure, reliable and industrial strength solution for its customers, which include solopreneurs, fast-growing startups, large multi-national companies and government agencies. The company is a strategic partner to Dassault Systémes, and today powers hundreds of organizations throughout North America, Europe and Asia. Outscale’s U.S. headquarters is located in Jersey City, New Jersey, and its global headquarters is in Saint-Cloud, France.
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
SYS-CON Events announced today that Interoute has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Interoute is the owner operator of Europe's largest network and a global cloud services platform, which encompasses over 70,000 km of lit fiber, 15 data centers, 17 virtual data centers and 33 colocation centers, with connections to 195 additional partner data centers. Our full-service Unified ICT platform serves startups and international enterprises, as well as every major European telecommunications service provider and major operators across the world, global internet giants, governments and universities.
SYS-CON Events announced today that Systena America will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Systena Group has been in business for various software development and verification in Japan, US, ASEAN, and China by utilizing the knowledge we gained from all types of device development for various industries including smartphones (Android/iOS), wireless communication, security technology and IoT services.
SYS-CON Events announced today that CollabNet, a global leader in enterprise software development, release automation and DevOps solutions, will be a Bronze Sponsor of SYS-CON's 20th International Cloud Expo®, taking place from June 6-8, 2017, at the Javits Center in New York City, NY. CollabNet offers a broad range of solutions with the mission of helping modern organizations deliver quality software at speed. The company’s latest innovation, the DevOps Lifecycle Manager (DLM), supports Value Stream Mapping for the development and operations tool chain by offering DevOps Tool Chain Integration and Traceability; DevOps Tool Chain Orchestration; and DevOps Insight and Intelligence. CollabNet also offers traditional application lifecycle management, ALM, for the enterprise through its TeamForge product.
Every successful software product evolves from an idea to an enterprise system. Notably, the same way is passed by the product owner's company. In his session at 20th Cloud Expo, Oleg Lola, CEO of MobiDev, will provide a generalized overview of the evolution of a software product, the product owner, the needs that arise at various stages of this process, and the value brought by a software development partner to the product owner as a response to these needs.
SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in compute, storage and networking technologies, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/Big Data, HPC and Embedded Systems worldwide. Supermicro is committed to protecting the environment through its “We Keep IT Green®” initiative and provides customers with the most energy-efficient, environmentally friendly solutions available on the market.
SYS-CON Events announced today that A&I Solutions has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 1999, A&I Solutions is a leading information technology (IT) software and services provider focusing on best-in-class enterprise solutions. By partnering with industry leaders in technology, A&I assures customers high performance levels across all IT environments including: mainframe, distributed, virtual, and cloud platforms. With expertise in the entire application lifecycle —Plan, Build, Test, Deploy, Operate, and Secure— A&I enables customers to transform their businesses by driving consistency and maximizing efficiency.
SYS-CON Events announced today that EARP will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. "We are a software house, so we perfectly understand challenges that other software houses face in their projects. We can augment a team, that will work with the same standards and processes as our partners' internal teams. Our teams will deliver the same quality within the required time and budget just as our partners' teams would do. It helps our partners to expand their operations with our forces."
SYS-CON Events announced today that Tappest will exhibit MooseFS at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. MooseFS is a breakthrough concept in the storage industry. It allows you to secure stored data with either duplication or erasure coding using any server. The newest – 4.0 version of the software enables users to maintain the redundancy level with even 50% less hard drive space required. The software functions on all major operating systems and is used around the world by businesses, universities and NGOs. The MooseFS team provides on-site implementation and support for your operations.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.