You can also hold down shift and click on the icon to open up a new window. Similarly if you use a keyboard shortcut to open your icons (like Win+1 to open the

You can also hold down shift and click on the icon to open up a new window. Similarly if you use a keyboard shortcut to open your icons (like Win+1 to open the first icon) holding shift while pressing the shortcut will open a new window (Shift+Win+1 will open a new window for the first icon).