5 ms·

What happens if the guest and host are both x86?

by rank0 4y ago

peter_d_sherman 4y ago
Well, then you would have to know: a) The specific x86 chip manufacturer and architecture (i.e., Intel, AMD, VIA, IDT, Cyrix, Transmeta, Zhaoxin, MCST (Elbrus), ? -- see https://en.wikipedia.org/wiki/List_of_x86_manufacturers https://en.wikipedia.org/wiki/List_of_x86_manufacturers -- there's a huge difference in potential security issues between say, the earliest Intel 386, and the latest Intel or AMD (or compatible) 64-bit x86 multi-cored multi-threaded CPU... b) What security issues are potentially present in that CPU; c) What the Virtual Machine / Emulator and/or Virtualized Environment does or doesn't do. (For example, if you use Bochs (https://bochs.sourceforge.io/ https://bochs.sourceforge.io/) for the guest environment on an x86 host -- then there's absolutely nothing that the guest OS can do to mitigate host OS security issues -- that's because all instructions inside of the guest are emulated -- security must be handled at the level of the OS/Hardware that is running Bochs, that is, at the level of the host.) Now, things get slightly more weird with Virtual Machines/Emulators/Virtual Environments that permit some/all of the x86 instructions to run directly on the host hardware. Here we have to distinguish between user-mode and "protected mode" (OS and/or Hypervisor, Ring < 3, instructions)... You see, a Virtual Machines/Emulator/Virtual Environments -- may permit patterns of the following sort: 1) Some/All of the user-mode x86 instructions in the guest environment to run directly in the host environment without translation/modification... 2) Some/All of the user-mode x86 instructions AND some/all of the "protected mode" (OS and/or Hypervisor, Ring < 3, instructions) to run directly in the host environment without translation/modification... Now, I am not an expert on any of this. I am not an expert on all of the Virtual Machines/Emulator/Virtual Environments out there, nor am I an expert on all x86 instructions, protected mode or user mode... But what I can tell you is simply this: What x86 instructions a Virtual Machine/Emulator/Virtual Environment permits to be run directly, that is, which x86 instructions, existing in the guest OS, ran directly on the host CPU (without modification or deletion or "hooking"/"intercepting" or what-have-you), could have a large impact on security... Which is a double-edged sword! The more power that is given to the guest OS to access the lower layers (the host or physical hardware) -- the more guest OS's can potentially mitigate security issues... But on the other hand: The more power that is given to a guest OS to access the lower layers (the host or physical hardware) -- the more that guest OS can potentially CAUSE security issues! It's a double-edged sword! Ultimately: The more you can know about the layers of your specific stack, the better the position you'll be in to control security, and security-related issues... Anyway, great question! I'm not the expert... take everything that I'm saying with the proverbial "grain of salt"... It's a good idea to do all of the research necessary to know your own specific stack... Also, one thing that I do know -- that if you're running an x86 emulator on a non-x86 CPU, the option (from the Virtual Machine/Emulator/Virtual Environment's point-of-view) of simply passing on instructions to the lower-level CPU for direct execution -- no longer exists -- in other words, running on a non-x86 CPU (i.e. ARM, RISC-V, etc.) -- the Virtual Machine/Emulator/Virtual Environment now MUST translate or interpret x86 instructions in the guest -- it can no longer ask the underlying host to run them directly, without modification... I know that was not your question -- but I think it's an important additional observation...
- rank0 4y ago
  I appreciate the detailed response! You certainly got the gears turning in my brain. A couple points: > What x86 instructions a Virtual Machine/Emulator/Virtual Environment permits to be run directly, that is, which x86 instructions, existing in the guest OS, ran directly on the host CPU (without modification or deletion or "hooking"/"intercepting" or what-have-you), could have a large impact on security... How would I even begin to map out which instructions get passed directly to the host cpu? I mean even if we just look at KVM, there's many other pieces of software that go into a virtualization environment. Wouldn't qemu, virsh, or KVM versions/configuration impact which instructions go right to the hardware? We haven't even gotten to the guest config, or any host hardware details! My instinct says it's extremely difficult to answer our question without a team of engineers. > if you're running an x86 emulator on a non-x86 CPU, the option (from the Virtual Machine/Emulator/Virtual Environment's point-of-view) of simply passing on instructions to the lower-level CPU for direct execution -- no longer exists -- in other words, running on a non-x86 CPU (i.e. ARM, RISC-V, etc.) -- the Virtual Machine/Emulator/Virtual Environment now MUST translate or interpret x86 instructions in the guest -- it can no longer ask the underlying host to run them directly, without modification... Makes sense to me. My follow up is now, does paravirtualization effectively create the same translation requirement? If I have an x86 xen-based hypervisor running x86 guests, does the virtual interface provided by xen give us a similar security control that ARM->x86 provides? (Or at least "good enough")?
  - peter_d_sherman 4y ago
    >I appreciate the detailed response! I appreciate the appreciation! But please remember I am not an expert in this area -- only a fellow interested party. A "student" (for lack of a better term) of sorts. I'll tell you everything I know -- but keep in mind that in this setting I am not unlike Hamlet's Horatio -- that is, "There are more things in Heaven and Earth -- that are dreamt of in my philosophy..." :-) >Wouldn't qemu, virsh, or KVM versions/configuration impact which instructions go right to the hardware? They would indeed! >How would I even begin to map out which instructions get passed directly to the host cpu? It's a great question! If I recall correctly, Bochs has/had a compilation option whereby if a flag was set, Bochs would to (at the expense of a lot of speed!) proceed to create a log or report or tallying of some sort -- such that when you exited it, it would show a list of each x86 instruction type that was used during runtime -- in other words, if you ran an OS in Bochs -- you could then get a list of ALL of the x86 instructions it used! Very handy if you're trying to get a new/alien OS to work on an x86 emulator, if/when in early versions of the emulator there might be bugs in emulating specific x86 instructions... QEMU might have (or have had!) the same facility. I am not a QEMU expert, so I don't know. Also, there's at least 3 possible things that can happen to any given instruction: 1) It can go through unmodified; 2) It can go through modified (for example, jumps may have their target address modified; this is because if code portions are expanded because of other modification, jump target locations will change...) 3) It can be intercepted by the VM environment (and/or Hypervisor), where specific code in the VM environment or Hypervisor is executed to replace or emulate the functionality of the instruction... QEMU might have a way to log/count instructions that it intercepts/modifies, albeit, if this exists, this would probably be a compilation-time switch that would need to be set, and performing this would probably slow down guest considerably... >My instinct says it's extremely difficult to answer our question without a team of engineers. My instinct says that your instinct is extremely correct! >My follow up is now, does paravirtualization effectively create the same translation requirement? Again, I'm not an expert. I would guess that it would not be off-limits for the paravirtualizer to engage in translation, but I would also guess that if the paravirtualizer could avoid it for the most part, it would... but those are only guesses... If you really want to understand "how deep the rabbit hole goes" (AFAIK) -- then I would start by going here: http://pascal.hansotten.com/uploads/pascals/pascal-s%20snepscheut.pas http://pascal.hansotten.com/uploads/pascals/pascal-s%20sneps... Now, what is this mess of code, you might ask? Well for this exercise, IGNORE MOST OF IT!. Do a word search for 'procedure interpret', of which part looks like this: procedure interpret; var pc, sp, j, k, n: integer; i: instr; c: char; h: boolean; begin pc:= 0; h:= false; repeat i:= code[pc]; pc:= pc+1; case i.op of add : begin m[sp+1]:= m[sp+1]+m[sp]; sp:= sp+1 end; neg : m[sp]:= -m[sp]; mul : begin m[sp+1]:= m[sp+1]*m[sp]; sp:= sp+1 end; divd : begin m[sp+1]:= m[sp+1] div m[sp]; sp:= sp+1 end; [...20 or so more lines deleted, for brevity...] end until h end; So what is that, and how is it relevant to this discussion you may ask! Ah! You see, that's a Turing-complete machine implemented 20 or so lines of Pascal Code! (in this case, inside of an early pascal compiler, the reason for which being that if you were running binary instructions (such as compiled output!) on an early computer without memory protection, you could easily crash the computer, but if you ran that same program inside of this emulation environment, then that could no longer happen!) This procedure, procedure "Interpret" (terrible name for it, IMHO) -- actually implements one of the first known and simplest Virtual Machines in existence! (There are simpler ones, but they're more academic, so we'll ignore them for now!). Basically that Virtual Machine, procedure "Interpret" -- could (in theory!) run any program that any of today's computers could, everything from DOOM to Spreadsheets to Operating Systems to what-have-you. Why? Because it's Turing-complete! But... so is the x86 chip! (And every other CPU out there!). SP, incidentally, which is used inside of that code -- can be thought of as x86's ESP (32-bit), or RSP (64-bit). It's a stack pointer, that's all it is! Similarly when you see 'm' and brackets [] -- that's just (virtualized!) memory! 'pc' (program counter) = x86's IP (16 bit)/EIP (32 bit)/RIP (64 bit). OK, so now with all of this background, hopefully you see how on the one hand how easy it is to create a Software Virtual Machine (not a hardware one, which is a different animal!) -- but on the other hand, parsing x86 instructions and doing so correctly might require pages and pages of esoteric code! Then we have the (hardware!) problem that later x86 CPU's allow multiple levels of nested virtualization -- that is, if those later x86 chips were designed correctly, then in theory you could have an almost infinite number of nested virtual machines (subject only to memory availability) -- and that's only in Hardware! A software virtual machine (such as procedure Interpret, or any other Turing-Complete software!) -- can in theory run an almost infinite number of nested (software) virtual machines! One inside the other! Subject only to CPU resource availability (the more you try to run, the slower they get, etc.) So between the two (hardware and software), you could have rabbit holes on top of rabbit holes... Which is why I claim "Socratic Ignorance" ("The only thing I know is that I know nothing"). And then of course there's the age-old philosophic question of "Who guards the guardians?" -- in other words, even though a virtualization environment (or any piece of software or hardware on the stack) can add security -- if it isn't properly written/debugged/audited -- it can take security away. In other words, "Who guards the guardians?" Anyway, it's a broad topic area. I'm not the "go-to guy" for this! But great questions!

What happens if the guest and host are both x86?

by rank0 4y ago

What happens if the guest and host are both x86?

Well, then you would have to know: a) The specific x86 chip manufacturer and architecture (i.e., Intel, AMD, VIA, IDT, Cyrix, Transmeta, Zhaoxin, MCST (Elbrus), ? -- see https://en.wikipedia.org/wiki/List_of_x86_manufacturers https://en.wikipedia.org/wiki/List_of_x86_manufacturers -- there's a huge difference in potential security issues between say, the earliest Intel 386, and the latest Intel or AMD (or compatible) 64-bit x86 multi-cored multi-threaded CPU... b) What security issues are potentially present in that CPU; c) What the Virtual Machine / Emulator and/or Virtualized Environment does or doesn't do. (For example, if you use Bochs (https://bochs.sourceforge.io/ https://bochs.sourceforge.io/) for the guest environment on an x86 host -- then there's absolutely nothing that the guest OS can do to mitigate host OS security issues -- that's because all instructions inside of the guest are emulated -- security must be handled at the level of the OS/Hardware that is running Bochs, that is, at the level of the host.) Now, things get slightly more weird with Virtual Machines/Emulators/Virtual Environments that permit some/all of the x86 instructions to run directly on the host hardware. Here we have to distinguish between user-mode and "protected mode" (OS and/or Hypervisor, Ring < 3, instructions)... You see, a Virtual Machines/Emulator/Virtual Environments -- may permit patterns of the following sort: 1) Some/All of the user-mode x86 instructions in the guest environment to run directly in the host environment without translation/modification... 2) Some/All of the user-mode x86 instructions AND some/all of the "protected mode" (OS and/or Hypervisor, Ring < 3, instructions) to run directly in the host environment without translation/modification... Now, I am not an expert on any of this. I am not an expert on all of the Virtual Machines/Emulator/Virtual Environments out there, nor am I an expert on all x86 instructions, protected mode or user mode... But what I can tell you is simply this: What x86 instructions a Virtual Machine/Emulator/Virtual Environment permits to be run directly, that is, which x86 instructions, existing in the guest OS, ran directly on the host CPU (without modification or deletion or "hooking"/"intercepting" or what-have-you), could have a large impact on security... Which is a double-edged sword! The more power that is given to the guest OS to access the lower layers (the host or physical hardware) -- the more guest OS's can potentially mitigate security issues... But on the other hand: The more power that is given to a guest OS to access the lower layers (the host or physical hardware) -- the more that guest OS can potentially CAUSE security issues! It's a double-edged sword! Ultimately: The more you can know about the layers of your specific stack, the better the position you'll be in to control security, and security-related issues... Anyway, great question! I'm not the expert... take everything that I'm saying with the proverbial "grain of salt"... It's a good idea to do all of the research necessary to know your own specific stack... Also, one thing that I do know -- that if you're running an x86 emulator on a non-x86 CPU, the option (from the Virtual Machine/Emulator/Virtual Environment's point-of-view) of simply passing on instructions to the lower-level CPU for direct execution -- no longer exists -- in other words, running on a non-x86 CPU (i.e. ARM, RISC-V, etc.) -- the Virtual Machine/Emulator/Virtual Environment now MUST translate or interpret x86 instructions in the guest -- it can no longer ask the underlying host to run them directly, without modification... I know that was not your question -- but I think it's an important additional observation...

rank0 4y ago
I appreciate the detailed response! You certainly got the gears turning in my brain. A couple points: > What x86 instructions a Virtual Machine/Emulator/Virtual Environment permits to be run directly, that is, which x86 instructions, existing in the guest OS, ran directly on the host CPU (without modification or deletion or "hooking"/"intercepting" or what-have-you), could have a large impact on security... How would I even begin to map out which instructions get passed directly to the host cpu? I mean even if we just look at KVM, there's many other pieces of software that go into a virtualization environment. Wouldn't qemu, virsh, or KVM versions/configuration impact which instructions go right to the hardware? We haven't even gotten to the guest config, or any host hardware details! My instinct says it's extremely difficult to answer our question without a team of engineers. > if you're running an x86 emulator on a non-x86 CPU, the option (from the Virtual Machine/Emulator/Virtual Environment's point-of-view) of simply passing on instructions to the lower-level CPU for direct execution -- no longer exists -- in other words, running on a non-x86 CPU (i.e. ARM, RISC-V, etc.) -- the Virtual Machine/Emulator/Virtual Environment now MUST translate or interpret x86 instructions in the guest -- it can no longer ask the underlying host to run them directly, without modification... Makes sense to me. My follow up is now, does paravirtualization effectively create the same translation requirement? If I have an x86 xen-based hypervisor running x86 guests, does the virtual interface provided by xen give us a similar security control that ARM->x86 provides? (Or at least "good enough")?
- peter_d_sherman 4y ago
  >I appreciate the detailed response! I appreciate the appreciation! But please remember I am not an expert in this area -- only a fellow interested party. A "student" (for lack of a better term) of sorts. I'll tell you everything I know -- but keep in mind that in this setting I am not unlike Hamlet's Horatio -- that is, "There are more things in Heaven and Earth -- that are dreamt of in my philosophy..." :-) >Wouldn't qemu, virsh, or KVM versions/configuration impact which instructions go right to the hardware? They would indeed! >How would I even begin to map out which instructions get passed directly to the host cpu? It's a great question! If I recall correctly, Bochs has/had a compilation option whereby if a flag was set, Bochs would to (at the expense of a lot of speed!) proceed to create a log or report or tallying of some sort -- such that when you exited it, it would show a list of each x86 instruction type that was used during runtime -- in other words, if you ran an OS in Bochs -- you could then get a list of ALL of the x86 instructions it used! Very handy if you're trying to get a new/alien OS to work on an x86 emulator, if/when in early versions of the emulator there might be bugs in emulating specific x86 instructions... QEMU might have (or have had!) the same facility. I am not a QEMU expert, so I don't know. Also, there's at least 3 possible things that can happen to any given instruction: 1) It can go through unmodified; 2) It can go through modified (for example, jumps may have their target address modified; this is because if code portions are expanded because of other modification, jump target locations will change...) 3) It can be intercepted by the VM environment (and/or Hypervisor), where specific code in the VM environment or Hypervisor is executed to replace or emulate the functionality of the instruction... QEMU might have a way to log/count instructions that it intercepts/modifies, albeit, if this exists, this would probably be a compilation-time switch that would need to be set, and performing this would probably slow down guest considerably... >My instinct says it's extremely difficult to answer our question without a team of engineers. My instinct says that your instinct is extremely correct! >My follow up is now, does paravirtualization effectively create the same translation requirement? Again, I'm not an expert. I would guess that it would not be off-limits for the paravirtualizer to engage in translation, but I would also guess that if the paravirtualizer could avoid it for the most part, it would... but those are only guesses... If you really want to understand "how deep the rabbit hole goes" (AFAIK) -- then I would start by going here: http://pascal.hansotten.com/uploads/pascals/pascal-s%20snepscheut.pas http://pascal.hansotten.com/uploads/pascals/pascal-s%20sneps... Now, what is this mess of code, you might ask? Well for this exercise, IGNORE MOST OF IT!. Do a word search for 'procedure interpret', of which part looks like this: procedure interpret; var pc, sp, j, k, n: integer; i: instr; c: char; h: boolean; begin pc:= 0; h:= false; repeat i:= code[pc]; pc:= pc+1; case i.op of add : begin m[sp+1]:= m[sp+1]+m[sp]; sp:= sp+1 end; neg : m[sp]:= -m[sp]; mul : begin m[sp+1]:= m[sp+1]*m[sp]; sp:= sp+1 end; divd : begin m[sp+1]:= m[sp+1] div m[sp]; sp:= sp+1 end; [...20 or so more lines deleted, for brevity...] end until h end; So what is that, and how is it relevant to this discussion you may ask! Ah! You see, that's a Turing-complete machine implemented 20 or so lines of Pascal Code! (in this case, inside of an early pascal compiler, the reason for which being that if you were running binary instructions (such as compiled output!) on an early computer without memory protection, you could easily crash the computer, but if you ran that same program inside of this emulation environment, then that could no longer happen!) This procedure, procedure "Interpret" (terrible name for it, IMHO) -- actually implements one of the first known and simplest Virtual Machines in existence! (There are simpler ones, but they're more academic, so we'll ignore them for now!). Basically that Virtual Machine, procedure "Interpret" -- could (in theory!) run any program that any of today's computers could, everything from DOOM to Spreadsheets to Operating Systems to what-have-you. Why? Because it's Turing-complete! But... so is the x86 chip! (And every other CPU out there!). SP, incidentally, which is used inside of that code -- can be thought of as x86's ESP (32-bit), or RSP (64-bit). It's a stack pointer, that's all it is! Similarly when you see 'm' and brackets [] -- that's just (virtualized!) memory! 'pc' (program counter) = x86's IP (16 bit)/EIP (32 bit)/RIP (64 bit). OK, so now with all of this background, hopefully you see how on the one hand how easy it is to create a Software Virtual Machine (not a hardware one, which is a different animal!) -- but on the other hand, parsing x86 instructions and doing so correctly might require pages and pages of esoteric code! Then we have the (hardware!) problem that later x86 CPU's allow multiple levels of nested virtualization -- that is, if those later x86 chips were designed correctly, then in theory you could have an almost infinite number of nested virtual machines (subject only to memory availability) -- and that's only in Hardware! A software virtual machine (such as procedure Interpret, or any other Turing-Complete software!) -- can in theory run an almost infinite number of nested (software) virtual machines! One inside the other! Subject only to CPU resource availability (the more you try to run, the slower they get, etc.) So between the two (hardware and software), you could have rabbit holes on top of rabbit holes... Which is why I claim "Socratic Ignorance" ("The only thing I know is that I know nothing"). And then of course there's the age-old philosophic question of "Who guards the guardians?" -- in other words, even though a virtualization environment (or any piece of software or hardware on the stack) can add security -- if it isn't properly written/debugged/audited -- it can take security away. In other words, "Who guards the guardians?" Anyway, it's a broad topic area. I'm not the "go-to guy" for this! But great questions!