I am trying to make a programming language for the first time and wonder how it handles whitespace. I can see CFG by default seem to just ignore whitespaces, but what if I want it to use it sometimes? Like when you declare an int people usually write "int b", and the whitespace separate these two items, but if I write something like "5 + 5" it should ignore the whitespace. Is that simply done by implementing in the rule for declarations like this:

Dcl --> int" " Id

Expr --> "int+int".

This should force a user to write a whitespace between the int and Id, but don't care whether there is a whitespace in the Expr rule. Is this correctly understood?

  • $\begingroup$ So what you are saying is that the grammar should ignore whitespace completely and I should make my parser acknowledge whitespace in certain situations, like with the "Int b" example? $\endgroup$ – Michael Mar 3 '18 at 11:34
  • $\begingroup$ That makes great sense. Thank you very much for your help! $\endgroup$ – Michael Mar 3 '18 at 11:59
  • $\begingroup$ As I've accidentally seem to have answered your question, I've replaced into 'an answer' below. Note that that is the place where answers should go: your post should only contain a question. $\endgroup$ – Discrete lizard Mar 3 '18 at 12:03
  • 1
    $\begingroup$ Ye I saw. I also edited the post before with the solution for all to find. But removed it again now that you've made it an answer and marked it as the answer. Thank you! $\endgroup$ – Michael Mar 3 '18 at 12:04
  • $\begingroup$ @Michael If you do post a question and then find out the answer, you're welcome (encouraged, even!) to post the answer to your own question, but it should go in the answer box, not as part of the question. $\endgroup$ – David Richerby Mar 3 '18 at 12:13

I think you're approaching this the wrong way. Writing spaces is better handled as a problem of parsing, not of defining your grammar. That is, your grammar should only concern itself with lists of 'tokens' and your parser is concerned with transforming text into 'tokens' properly.

In a sense, whitespace is something the grammar should be completely unaware of. Whitespace can be useful for your parser to determine whether something is one or two tokens, but I doubt that you want to consider whitespace as a token by itself.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.