Re: [請益] 想不通直譯器vs編譯器vs機器碼的問題

看板Soft_Job作者snaketsai (さいでんし)時間7年前 (2018/05/16 14:21)推噓4(4推 0噓 15→)

留言19則, 6人參與討論串9/9 (看更多)

Update: 睡前簡單弄個小更新把我回文中說的code cache的hashtable快速弄出來了第一次呼叫時會emite code，第二次因為hashtable查找成功則直接呼叫另外就是把mmap要記憶體時的可讀可寫可執行，改成僅可讀可寫後面再透過mprotect改成可讀可執行這樣可以防止後來有人竄改JIT emit的code cache patch放在這邊： https://paste.plurk.com/show/2636452/ P.S. 我不用github放的原因是因為我不想暴露我的主ID 還請擔待 = = = = = = = = = = = = 中午吃飽飯有點脹氣看到某篇回文的推文中有人想要看JIT範例所以簡單寫了個很粗糙的版本前後花不到15分鐘所以覺得coding style很爛、沒效率是很正常的，別打我QQ (結果真的被轉出去了Orz 我晚點忙完會來修code 真的assembly code gen會找時間補上的）大抵上的思路就是你吃到虛擬機的bytecode，就把它轉成host平台的native code 接著把它塞進你跟系統動態要的、可以執行的記憶體區段然後就很快樂的開始執行它黃色上色的地方，每個作業系統給的API不一樣 Linux、OSX、FreeBSD ... (most of *NIX OSes)： mprotect()、mmap() with proper permissions. Windows： VirtualAlloc() = = = = = = = = = = = /** * * This piece of code is to demo simple VM/JIT knowledge * and is released under 2-clause BSD license. * * Copyright 2018/05/16 snaketsai * Redistribution and use in source and binary forms, with or without modification, * are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * U2FsdGVkX19w8Ikk1T7xBlbh4vDhIEvZzshUhXft6XMFugC9M27uV9LDszf7/8gP * OtF2AZwYaUQqzLLY5vXhCQ== * **/ #include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/mman.h> #ifdef _NOJIT static inline int IAdd(int inp1, int inp2) { puts("Non-JIT version iadd."); return inp1+inp2; } static inline int ISub(int inp1, int inp2) { puts("Non-JIT version isub."); return inp1-inp2; } #else // Nope, I'm not gonna write an assembler. // Just pretend that we _magically_ get assembly pieces we need. const unsigned char _add[] = \ "\x55\x48\x89\xe5\x89\x7d\xfc\x89\x75\xf8\x8b\x55\xfc\x8b\x45\xf8\x01\xd0\x5d\xc3\x00"; const unsigned char _sub[] = \ "\x55\x48\x89\xe5\x89\x7d\xfc\x89\x75\xf8\x8b\x45\xfc\x2b\x45\xf8\x5d\xc3\x00"; #define MAXCODECAHE_SIZE 4096 void* AllocExeMem(size_t size) { void* memPtr = mmap(0, size, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (memPtr == (void*)-1) { perror("mmap() failed."); return NULL; } return memPtr; } void *CodeCachePool; int (*IAdd)(int, int); int (*ISub)(int, int); #endif /** Very Simple VM spec - - Every insn. is 3 byte long. opcodes are listed below: 0x00 HCF, halt and catch fire. 0x01 IADD, humbly add two signed integers and print the result. 0x02 ISUB, humbly substract two signed integers and print the result. The bytecode stream should end with a 0xff byte mark. **/ const unsigned char BytecodeStream[] = \ "\x01\x02\x03\x02\x05\x04\x00\x00\x00\xff"; int runVM(const unsigned char* bstream) { unsigned char insn[3]; while(bstream[0] != 0xff) { memcpy(&insn,bstream,3); switch(insn[0]) { case 0x00: puts("Dave, stop, will you ?"); return 0; case 0x01: #ifndef _NOJIT // emit code to code cache. memcpy(CodeCachePool, _add, sizeof(_add)); #endif printf("iadd: %d\n", IAdd((int)insn[1], (int)insn[2])); break; case 0x02: #ifndef _NOJIT memcpy(CodeCachePool, _sub, sizeof(_sub)); #endif printf("isub: %d\n", ISub((int)insn[1], (int)insn[2])); break; default : // Unrecognized insn. perror("Sorry Dave, I can't do that."); return -1; } bstream+=3; } } int main(int ac, char* av[]) { int ret = -1; #ifndef _NOJIT CodeCachePool = AllocExeMem(MAXCODECAHE_SIZE); IAdd = CodeCachePool; ISub= CodeCachePool; #endif ret = runVM(BytecodeStream); if(ret == 0) { perror("VM exited successfully."); } else { perror("Unexpected error occured."); } return 0; } -- Linux is the bone of my world. Kernel is mybody, and initramfs is myblood, have created over a thousand Distros. Unknown to impossibility. Nor known to limitation. -- ※ 發信站: 批踢踢實業坊(ptt.cc), 來自: 140.113.167.181 ※ 文章網址: https://www.ptt.cc/bbs/Soft_Job/M.1526451691.A.2E2.html ※ 編輯: snaketsai (140.113.167.181), 05/16/2018 14:58:35

推

cklppt

05/16 18:51, 7年前 , 1^F

05/16 18:51, 1^F

推

LinuxKernel

05/16 19:56, 7年前 , 2^F

05/16 19:56, 2^F

這整份code其實沒什麼技術背景，大概上過compiler課的人都會寫唯一需要額外知道的，是如何去要一塊可塞code又可執行的記憶體這個課堂上不會教

推

soheadsome

05/16 19:58, 7年前 , 3^F

05/16 19:58, 3^F

→

weiyucsie

05/16 20:23, 7年前 , 4^F

05/16 20:23, 4^F

沒欸，會那麼長是因為包含function call要的prologue跟epilogue 我其實弄得比較像intrinsic fnuction 所以Non-JIT的部份可以直接跟JIT的無縫接在一起

→

weiyucsie

05/16 22:22, 7年前 , 5^F

05/16 22:22, 5^F

沒想到有這麼完整的write up 馬上google一下"mprotect/mmap"+"JIT"發現已經不少文章在講我第一次知道JIT大概是2011年，當時文章還真的不多稍微說一下我這篇目前離實用code的差距在： (1) 缺真正的asm codegen，我上面的asm snippet是用gcc -c去編C code來生的 (2) JIT emit code後沒有code reuse、每次都還會重新emit 實務上會是個hashtable，查找失敗emit、查找成功直接call 更進階一點會有所謂的block chaining，也就是把能接在一起的native code串在一起跑這樣可以節省跳進、又跳出VM code的時間 (3) mmap權限直接開executable其實有點危險，比較正確的作法會分兩段：先mmap開rw要記憶體、把code塞進去然後用mprotect把它改設成rx ※ 編輯: snaketsai (140.113.167.181), 05/16/2018 23:24:51

→

weiyucsie

05/17 00:05, 7年前 , 6^F

05/17 00:05, 6^F

→

weiyucsie

05/17 00:06, 7年前 , 7^F

05/17 00:06, 7^F

→

weiyucsie

05/17 00:07, 7年前 , 8^F

05/17 00:07, 8^F