Data Visualization

Summary

The modes are chosen based on RD cost, which is a function of distortion and coding rate.

Each CU will choose the optimal mode which has the least RD Cost.

There might be a chance to choose the mode with distortion but lower coding rate.

The distortion would also depend on the distortion of synthesized texture view if VSO is on.

Memo

Note

Please notice this visualization is for the data without any pre-processing, i.e., the data that we obtained right after encoding the video sequences. You can get a sense of what kind of data that we are dealing with. And you might know the reason for the techniques executed in the pre-processing steps, such as edge strength analysis for smooth-removing.

In the python project for pre-processing the data, which is data-processing-for-fast-depth-coding, you can visualize the collected data by typing commands from terminal:

python data_visu/visualize_blocks.py --block_size=32 --mode=2

Figure 0-0. running the visualization

Figure 0-1. Illustration of Intra modes [0, 34] and Figure 0-2. Examples of 8x8 luma prediction blocks generated with all the HEVC intra prediction modes. are the illustrations of Intra modes in HEVC in case you want to refresh your memory. (DMMs are not illustrated here).

Other figures are the visualization of some collected data.

Note

Posting all the visualization images here would be possibly overwhelming and unnecessary. However, if you want to see all the visualizations, that is also doable (by spending a little more time running the python script then capturing screen). Please just let me know if there’s a need :D

Important

Please compare Figure 2-2. Intra Mode: 2, Block Size: 16x16 with Figure 2-34. Intra Mode: 34, Block Size: 16x16. Compare the two blocks in red circle. You should be able to find the patterns are very similar to each other while their modes are totally different. This is the reason why we need to use TOP-5 accuracy instead of TOP-1 for evaluating the machine learning model.

TOP-5

To compare models, we examine how often the model fails to predict the correct answer as one of their top 5 guesses – termed “top-5 error rate”.

Tip

Feel free to click on the figures for inspecting their features carefully by enlarging or downloading. They are licensed under MIT License.

Intra modes in HEVC

Figure 0-1. Illustration of Intra modes [0, 34]

Figure 0-2. Examples of 8x8 luma prediction blocks generated with all the HEVC intra prediction modes.

Examples for block size 8x8

Figure 1-0. Intra Mode: 0, Block Size: 8x8

Figure 1-1. Intra Mode: 1, Block Size: 8x8

Figure 1-2. Intra Mode: 2, Block Size: 8x8

Figure 1-3. Intra Mode: 3, Block Size: 8x8

Figure 1-4. Intra Mode: 4, Block Size: 8x8

Figure 1-5. Intra Mode: 5, Block Size: 8x8

Figure 1-6. Intra Mode: 6, Block Size: 8x8

Figure 1-7. Intra Mode: 7, Block Size: 8x8

Figure 1-33. Intra Mode: 33, Block Size: 8x8

Figure 1-35. Intra Mode: 35, Block Size: 8x8

Figure 1-36. Intra Mode: 36, Block Size: 8x8

Examples for block size 16x16

Figure 2-0. Intra Mode: 0, Block Size: 16x16

Figure 2-1. Intra Mode: 1, Block Size: 16x16

Figure 2-2. Intra Mode: 2, Block Size: 16x16

Figure 2-3. Intra Mode: 3, Block Size: 16x16

Figure 2-34. Intra Mode: 34, Block Size: 16x16

Examples for block size 32x32

Figure 3-0. Intra Mode: 0, Block Size: 32x32

Figure 3-1. Intra Mode: 1, Block Size: 32x32

Figure 3-2. Intra Mode: 2, Block Size: 32x32

Figure 3-3. Intra Mode: 3, Block Size: 32x32

Figure 3-30. Intra Mode: 30, Block Size: 32x32

Figure 3-33. Intra Mode: 33, Block Size: 32x32

Figure 3-34. Intra Mode: 34, Block Size: 32x32

Figure 3-35. Intra Mode: 35, Block Size: 32x32

Figure 3-36. Intra Mode: 36, Block Size: 32x32

Examples for block size 64x64

Figure 4-0. Intra Mode: 0, Block Size: 64x64

Figure 4-1. Intra Mode: 1, Block Size: 64x64

Figure 4-2. Intra Mode: 2, Block Size: 64x64

Figure 4-3. Intra Mode: 3, Block Size: 64x64

Figure 4-4. Intra Mode: 4, Block Size: 64x64

Figure 4-5. Intra Mode: 5, Block Size: 64x64

Figure 4-32. Intra Mode: 32, Block Size: 64x64

Figure 4-33. Intra Mode: 33, Block Size: 64x64

Figure 4-34. Intra Mode: 34, Block Size: 64x64