Christopher Browne wrote:
> Weslee Bilodeau <weslee.bilodeau at hypermediasystems.com> writes:
>> Small question on this.
>>
>> I would imagine one could easily find a lot of edge cases that can break
>> the current parser.
>>
>> I'm guessing they found them in psql, which is why psql stole the lexer
>> from the backend itself.
>>
>> Is there any reason why slony didn't go the same route?
>>
>> http://developer.postgresql.org/cvsweb.cgi/pgsql/src/bin/psql/psqlscan.l?rev=1.21;content-type=text%2Fx-cvsweb-markup
>>
>> It was written for the exact same task -
>>
>> ------------------------------------------------------------
>> This code is mainly needed to determine where the end of a SQL statement
>> is: we are looking for semicolons that are not within quotes, comments,
>> or parentheses.  The most reliable way to handle this is to borrow the
>> backend's flex lexer rules, lock, stock, and barrel.  The rules below
>> are (except for a few) the same as the backend's, but their actions are
>> just ECHO whereas the backend's actions generally do other things.
>> ------------------------------------------------------------
>>
>>
>> Or has this already been discussed and dismissed?
> 
> If memory serves, I initially tried to introduce a second parser, but
> ran into trouble with the notion of linking in 2 parsers.
> 
> I then reviewed syntax, and figured I had enough of it covered, which
> was apparently not correct.
> 
> I'll take another crack at it...

I figure the $_$, $$, etc edge-casees would be another fun one to roll
into a custom parser.

CREATE FUNCTION test( ) RETURNS text AS $_$ SELECT ';', E'\';\'',
'"";""', E'"\';' ; SELECT 'OK'::text ; $_$ LANGUAGE SQL ;

SELECT $_$ hello; this ; - is '\" a '''' test $_$ ;

SELECT $$ $ test ; $ ;  $$ ;

All really funky, but perfectly valid.


And yeah, rolling two lexers into one, that does have its own challenges.

Maybe expand the first lexer and add an SQL state, so it can parse SQL
within the first one directly?


Weslee