Web Site UI Testing

"We were only changing the UI, so we don't need a full regression run here." - They said.

In Gerald Weinberg's "Secrets of Consulting" there was a wonderful chapter on "lullaby words". And "only" was one of them. Each time testers hear the word "only" - they start to get nervous. Or rather they smell bugs in the air :-)

I want to share my outlines that should be taken into account while testing "only UI" of a website (disregarding its platform or applied area).

Like any other testing technique, the user interface testing can be static (without running the code) and dynamic (with running the code).

Arbitrary static testing

Now it's nearly impossible to think of a site that you can observe without running a single line of JavaScript. But we can assume that checking that the layout design corresponds to requirements we do a static testing, because we are not interested in features and code, but only in the appearance. 
A tester just takes a png/psd/printout from a graphic designer and compares it to the page under test. You do not need your imagination here, just the attention.
This kind of testing is what "ordinary people" mean when they say "testing".

Dynamic testing

As soon as we want to test something that does not apply to layout design, we may consider this a dynamic technique (not only arbitrary, because the code is actually executed). All my further points are kinds of dynamic techniques and typically gray-box ones, as they access to the code or the data base.

Business Logic

The main part here is a business value, namely the system features which brought our system to existance, that are the reasons of why the system has been created. But each system has its own features, so you are free to fill this paragraph by yourself :-)

Client-server architecture

When we talk about a web site - it is always a client-server architecture.
Why should a tester care? Understanding the architecture we may insert our test data to the very levels where it can more accurately locate or even find a new bug.
In order to evaluate an appearance on the UI, let's see what elements of the system are participating in visulization:

•     server
•     browser
•     users with their data

Testing data on the server

With "test data" hereinafter, first of all I mean sets of character strings, such as the List of naughty strings , etc.

Test data have to be in the system during testing. On the "server" you can put them to the following places:

•     the database,
•     external files that our system processes at runtime,
•     API of the third-party services that our system may use to run.

User's Test Data

The test data that users can populate or adjust are as follows:

•     fields for user input;
•   simple input fields that we want to test
•     system configuration, available to the user;
•   eg. how many rows to display in any list, how often to clean the cache and all other settings that might affect the features look like
•     personal data and account settings;
•  The length of a user name or a nickname or an email may affect for example the reports or logs appearance
•     change layout options available to the user.
•  all kinds of user adjustable dash-boards, personal page content managers, enabling/disabling menus/functions/modules

Browser Window Size

Obviously the variables available to the user include the browser size. I would like to highlight the following dimension conbinations:

• Desktop
• Mobile / Responsive
• Something between those two
• Ve-e-ery widescreen / full screen on a large monitor.
• The narrow height to revise the operation of a vertical scroll bar; this is useful when testing long lists with infinitive scrolling.

Cache Levels

Any feature of your system may be coated with caches. The main drawback of using caches is that unlike other system components it is often is turned off on test environments. Therefore, when the code is rolled out to prod, you can observe unexpected effects.
We must know what kinds of cache are on production and when it is better to turn it on on the test server and not to miss an important bug.
I'll try to describe generally widespread types of cache that may be in most systems.

Browser

If you need to test a fresh user session use the incognito browser mode. But do not overdo it if you are interested in not only new, but also old users suuport who already have a set of cached resources.

• JS and CSS

        Both of these file types in most of the modern frameworks may be subjected to:

•  aggregation, collecting and combining of multiple files to reduce the number of network connections and speed up page loading
•  minification, removal of all symbols that do not affect the syntax from the source code: extra spaces, end-of-line symbols, extra semicolons etc. These changes make the code almost unreadable.

•  The browser caches minificated-aggregated files, as the server passes them.

•  Media Content

•  Pictures and other media are also cached.

•  Local Storage

• some of the information used in scripts on the page may be stored in a local repository (almost like a cookie, but with enhanced functionality. See More. on wiki Web_Storage )
• I have had problems myself with this in Jira in Firefox. Until I found how to clean it, pages were not loading up and stuck. It was a continuous nightmare!

Web Server

Web servers can cache some queries, returning the same response. Thus code is not executed. Be careful and carefully review the applied settings and the user guide for your web server.

Application Cache

Whatever framework has been used to build your site, it has its own way to cache up what is needed and what is not.

•  Functional cache.
•  At this level, too, there is some cache for JS + CSS + HTML of each feature/region/module.
•  Caching individual pages
•  Some frameworks have their own tools allowing to cache certain pages fully. These tools, in their turn, can exclude certain modules on cached pages from caching. Tangled :-) But it is important to realize that it's possible.

A fairy tale: Once upon a time, there was a high-loaded page that has been cached forever with all kinds of cache that occur in nature (on the framework level). And a very brave module was born at that time. Although it was very small looking with the only line of text somewhere in the middle of the page, but that line was calculated in a very interesting way even with a direct request to the database. And it worked, and there was no trouble to anybody until they met, the page and the module. And the site dropped down. And there was a day of debugging and roll-back. And then it turned out the module was not that simple but so proud that it had an explicit mark "never to cache" (for very special reason, perhaps the developer could not make it as special in another way). And a marketing department looked at this, and refused that very specical feature. And they split again, the page and the module.
 
The moral: Sometimes load-testing should be included in the regression test suite.

•  Database Cache
•  This is a very unexpected thing for me. DB cache is surely always there. But bugs associated with this type of cache, I had seen only twice, and I do not remember what they were :-) So I suspect that when someone references to the DB cache as the reason for the strange behavior, it's a lame excuse. The code that is written well (or even "correctly"?) does not deal with the DB cache, it has to be an internal affairs of your DBMS. (This is only my imperical feelings, and it does not apply to special systems with fine tuning).

AJAX

Last but not least, aspect of modern GUI testing are asynchronous requests to the server. With very these requests the whole modern business logic of the system is constructed and made available to users.
As it makes no sense to describe the business logic abstractly, here are some tips that should be kept in mind while testing pages rich with AJAX:

• What influences what?
• Build your own dependency graph of all requests on your page. (First of all you need to know which controls are static and which are gotten on demand)
• Imagine that one of the requests failed.
• ...
• PROFIT !!!11
• Make several successive iterations requesting the update of dependent UI controls. At some point it must break.
• Asynchronity
• The response time may appear to be quite long. Are others in the request queue ready to wait that much?
• What if the long awaited request do not get a reply at all?
• "On demand".
• Deliberately do not send a request.
• Or send the same one repeatedly.

Generally speaking, each end-point used in AJAX should be covered by the API testing (including automated checks) before you start testing the UI that depends on it. In manual API testing the Postman app helps me a lot. But testing of an API is a separate topic.

This is it for UI testing of a site! Let me know if I forgot something!