I wrote regexes for all the tokens in my programming language. How can I scan a piece of text for all of them? For instance, with these regexes:
int = `[0-9]+`
float = `[0-9]+(\.[0-9]+)?`
string = `"[^"]+"`
char = `'[^']'`
comment = `--(.+)?`
And this text:
12.3 12 "hey" 'y' -- Comment!
The output should be {float at 1 - 5, int at 6 - 8, string at 9 - 14, char at 15 - 18, comment at 19 - 30}. Is there a standard library function or a third-party library that can help me achieve this?