Join Stack Overflow to learn, share knowledge, and build your career.
My company's proprietary software generates a log file that is much easier to use if How To Write A Language Parser is parsed.
The log parser we all use was written by another employee as a side project, and it has horrible performance. These log files How To Write A Language Parser grow to 10s of megabytes very quickly, and the parser we currently use has issues if a log file is bigger than 1 megabyte. So, I want to write a program that can parse this massive amount of text in the shortest amount of time possible.
We use Windows exclusively, so running on Windows is a must. Our current implementation runs on a local web server, and I'm convinced that running click as an application would have to be faster.
My ultimate goal is to parse the text and display it in a much more user source manner with colors and such. Can you do this with Perl and Python? So, it will function like Notepad where you open a log file, but on the screen you display the user-friendly format instead of the raw file. So, I cant choose the best answer, and that was to choose a language that can best display what I'm going for, and then write the parser in that.
I changed the original question, since I guess I didn't ask what I was really looking for. Hmmm, "go with what you know" was a good answer.
How to write a Parser in C#? [closed] that Lexical analysis is ALSO what you'll want to understand if your going to write language/script parsers too. Writing a parser. Writing a parser is, depending on the language, a moderately complex task. In essence, it must transform a piece of code (which we inspect by. I'll show how I built a parser for a simple template language I How to build a parser it's surprisingly easy to hand-write a parser for a small language in. A few months ago I began a personal project to learn a bit more about how parsers and compilers work. to write a parser and language to write a. What language should I use to write a text parser and display the results a language that can best display what I'm going for, and then write the parser in.
Perl was designed for this sort of thing but imo is well suited for simple parsing, but I'd personally avoid it for complex projects. I personally find it tedious to hand-craft parsers, and even more tedious to debug them, but AntlrWorks is a lovely IDE which really makes it a piece of cake If you mess up your grammar rules, you will be informed. This is not the case with hand-crafted parsers, where you just scratch your body part and wonder about the "strange results" Even if you think your project is trivial now, it may well grow.
You should use the language that YOU know Unless you have so much time available to complete the project that you can also spend the time learning a new language. I would suggest using Python or Perl. Parsing large text files with regular expressions is really fast.
I could tell you that any macro assembler will let you write code that would rip through your data, but seriously, are you going to spend months writing assembly just to save a few seconds Professional Overview Resume CPU time? Rewriting a program is fun but it's not practical.
Whip out your profiler, point it at your horribly performing log parser, and fix the performance problems. If it's a common language, there will be people here who can help.
I've used both Python and Perl. Perl is a more natural fit for this but can be hard to maintain. Python will do it just as well and is easier to read. Maybe a finished product such as the MS LogParser usage podcast here may do what you need and it's free.
It's a special-purpose language, compatible with C, that's designed exactly for high-speed parsing of log files and other ad hoc data formats. There's even a feature where it can try to learn your log format from examples, although I don't know if that has hit production yet.
The people behind the project are really smart, and it's had a big impact within the phone company. PADS gives very high performance on data streams that produce gigabytes. Joe Bob says check it out. Continue reading "massive text in the shortest time possible ", Perl and Python are not the answer.
Tems of megabytes is not actually How To Write A Language Parser big. A number of very good text processing programs have been written in Perl. Ack a grep replacement is one. Sounds like a job for Perl, much as I don't particularly care for it as a language myself.
ActivePerl is a reasonable distribution of Perl for Windows. It How To Write A Language Parser practically built for parsing log files. I'd parse out the data using regular expressions, then use Template:: Use fscanf with care, since no types are checked, etc, it can result in errors.
All suggestions will be helpful. HenryAdamsJr 2 5 Some log sample would be nice, as well as how would you like it parsed. As for how I want to parse it, I've basically described that in my edit above. As for the log file itself, I don't need help with the parsing, just in choosing the best tool to do it.
You should probably also pick a language where displaying the text the way you want it is easy. Displaying might be more involved than the parsing itself.
You'll be able to get the text parsed no matter what you pick. Why would running as an application be faster that running through a web server? If it gets even a little complex, why not use a How To Write A Language Parser syntax and grammar set-up? That bit at the bottom is defining a grammar rule.
Willi Mentzel 5, 11 30 David 1 3. That's a great suggestion, and if this was needed within a certain timeframe, I would agree, but I was going to use this project as Resume Senior Relationship Manager excuse to learn something new. Reading all of the answers makes it seem like the language isn't going to make this faster or slower to a great extent.
Whatever language your coworker used. It wouldn't save a few seconds. If I do it right, it will literally save minutes.
With the current implementation, if the file is sufficiently big, it won't return at all. I feel that his implementation is wrong from the ground How To Write A Language Parser, and I don't have access to the source code, anyway. There's nothing like coming back to lines of symbol soup months later trying to figure out what the heck you were thinking. I added some information to the post to clarify how I'm going to be using the parsed text.
I want to have a GUI that will display the log, but in a friendly link. That just means you get to have all the fun twice!
I believe perl is considered a good choice to parse text. Gratzy 7, 4 19 Splunk is a popular log parsing and analysis tool that can accept arbitrary input: Parse this massive amount of text in the shortest time possible.
Norman Ramsey k 48 Http://cocktail24.info/blog/esl-dissertation-hypothesis-ghostwriting-website-gb.php converts the code into a C library. Perl is good for text processing. Matthew S 4. Would that be balanced out by how it can easily create a Windows GUI, or not?
No, it does not. Almost every other languages includes these as part of the language. I think this is both. It's not a discussion about being good or bad.
Sign up or log in Sign up using Google. Sign up using Facebook.
Sign up using Email and Password. Post as a guest Name.
Let's write a recursive-descent parser in C (Part 1)