Two years ago I posted an issue to greenplum project¹, targeting the long-known issue of poor performance of writable external table.

I wrote this piece of code to change the bufsize to 16M to reduce number of calls to 'curl_easy_perform', hence improving WET performance.

diff --git a/src/backend/access/external/url.c b/src/backend/access/external/url.c
index a81e687..1aaa6ba 100644
--- a/src/backend/access/external/url.c
+++ b/src/backend/access/external/url.c
@@ -2295,7 +2295,9 @@ static size_t curl_fwrite(char *buf, int nbytes, URL_FILE* file, CopyState pstat
         */
        if(!curl->out.ptr)
        {
-               const int bufsize = 64 * 1024 * sizeof(char);
+               const int bufsize = 16 * 1024 * 1024 * sizeof(char); /* Enlarge the bufsize to reduce call to curl_easy_perform,
+                                                                       hence WET performance should be improved.
+                                                                       The size itself is magic */
                MemoryContext oldcontext = CurrentMemoryContext;

                MemoryContextSwitchTo(CurTransactionContext); /* TODO: is there a better cxt to use? */

Gpfdist's buffer size should also be enlarged by using options -m 33554432.

The issue was then closed after a more robust patch accepted by greenplum team, the patch uses a GUC instead of hard-coded magic number to specify the buffersize.

At that time the source code of Greenplum has just been released to public, and the binaries I have had was different with the source released.

Instead of recompiling the source code to verify my code, I have to use this process to prove my findings:

Yao Yandong made a more robust patch according to my findings. Instead of hard-coded bufsize, the patch introduced a new GUC letting DBA to adjust this size. And the patch was merged to Greenplum when 4.3.7.1 was released. ²

This story happend in real world, but the customer name is faked.

One of my customers, The Big Customer(in abbrev, TBC), is running our product.

One day they come to me describe the following wierd phenomeon:

They observed a lot of ip packets send from containers in one component of our product, which is a virutal machine of a cluster, to other servers without doing SNAT(MASQUERADE), the receiving server is not a member of the cluster. As a result, these packets are then getting bounced into TBC’s core network, and triggering alerts.

These containers are hosting user applications, these containers are using internal ip addresses (usually 10.254.x.x). When processes inside these containers are going to connect outside servers, the packets need to be SNATed by the hosting virtual machine, this is done by a MASQUERADE rule in iptables’ “nat” table.

On the problematic system, a lot of packets with TCP flag FIN set get send out from container with source ip unchanged, the MASQUERADE rule does not take effect. When these packets reach the server, they will be considered invalid packets due to source address unrecognized, hence RST packets are emitted towards those source addresses, in this case, the 10.254.x.x addresses, and these packets get routed into TBC’s core network and trigger alerts.

As stated by TBC’s administrator of their core networking, they have been receiving these weird alerts for a long time, and only recently can they traced down the origin of these packets, which is send by virtual machines of our product.

The implicit release of response object is the root cause of this issue, this behavior will introduce an interval between service side close and client side close. This interval is longer than the virtual machine’s nf_conntrack_tcp_timeout_close_wait, and the conntrack entry will be discarded before client sends out FIN packet, hence this FIN packet will be marked as invalid and won’t be masqueraded. Suggestions on mitigating this problems are:

• improve code quality, avoid such implicit release of resources
• or an iptables rule like iptable -A FORWARD -j DROP -m conntrack --ctstate INVALID to drop each invalid packets.

Packrat parsing algorithm is an algorithm for parsing PEG¹, it can parse PEG in guaranteed linear time.² There are lots of resources about PEG and packrat algorithm here: http://bford.info/packrat/. Tony Garnock-Jones had an implementation of packrat in his blog, and he then rewrote it using MzScheme and then upon the parser he wrote a JSON read-write library. He also wrote a document about how to use it. This packrat library is included in Chicken Scheme's egg system³ and then be ported as a R7RS library⁴.

PEG is very similar to regexp. Here is the comparison of concepts between PEG and regexp

Generally PEG is more powerful than regular expressions, for example, PEG can handle nested parenthesis while regexp can't, this is because PEG is recursive. For example, regular expressions can't process JSON data with arbitrary depth.

Scheme has a powerful macro system, in most scheme implementation, only very limited primitive forms are implemented and others are derived by using macros. For example, r5rs standard defines 23 syntactic constructs, 11 of them can be defined using other more fundamental ones. ¹

There are many macro systems exist in Scheme's world, define-macro, syntax-rules, syntax-case are commonly adopted.

define-macro AKA defmacro is the traditional way of defining macro in lisp world. Macros defined by define-macro are roughly procedures transform(expand) input form to another s-expression, the transformation itself usually be called "applying the macro". Generally speaking, the body of define-macro can be any expression which returns a valid scheme expression on evaluation, and the arguments of define-macro are also the arguments of the expression(if seen as body of a lambda).

For example, this macro:

(define-macro (when cond exp . rest)
  `(if ,cond
       (begin ,exp . ,rest)))

will define a macro named when. And on applying this macro, such form (when (> n 0) exp1 exp2) will be transformed (expanded) as:

(if (> n 0)
    (begin exp1 
	   exp2))

This transformation done by macro when is just filling out a template using parameters of when: (> n 0), exp1 and (exp2). Or say, the macro generates new codes.

Because the nature of macros, code generation, macros are very powerful tools in lisp languages as well as in Scheme.

The define-macro macro system is so rough that it does not capture binding of any symbols appeared in macro definition, and these symbols' meaning can be changed when the macros applying, this will make the macros' behavior be very wierd if they are used with careless.

Consider the macro when defined in the previous secrtion, in the scenario demostraded here:

(let ((begin list))
  (when #f 1 2))

the code will be equivalent of:

(if #t
    (list 1 2))

This is because the define-macro only defines the form that when will be expanded but not the binding of symbols, these symbols' meaning can be altered in different context.

Such problem is called the hygienic problem.

Scheme implatations which have define-macro system also provide gensym or similar procedures which can generate symbols on the fly to mimic(partially) the hygienic problem of the define-macro system. For example, this code defined a naive macro swap! which swaps value of two variables:

(define-macro (swap! o1 o2)
  `(let ((tmp ,o1))
     (set! ,o1 ,o2)
     (set! ,o2 tmp)))

If one uses this marco like (swap! tmp something) or (swap! something tmp), the symbol tmp will be captured and the result is incorrect. This macro can be rewritten using gensym:

(define-macro (swap1 o1 o2)
  (let ((tmp (gensym)))
    `(let ((,tmp ,o1))
       (set! ,o1 ,o2)
       (set! ,o2 ,tmp))))

In this version of swap!, the danger of unwanted symbol capture will not exist.

syntax-rules is a solution to hygienic problems. Since it is adopted in the R5RS, it is wildly implemented in different Scheme implementations. syntax-rules uses pattern matching to construct macros.

With syntax-rules, the macro when can be re-written as:

(define-syntax when
  (syntax-rules ()
    ((when cond exp1 ...)
     (if cond
	 (begin exp1 ...)))))

The syntax of using syntax-rules is (syntax-rules literals (pattern template) ...), the macro will replace evey matching pattern to the corresponding template.

The major difference between syntax-rules and define-marco is that syntax-rules garentees hygienicness, in fact, all symbols occure in template has definite meaning: if the symbol is matched in pattern, the pattern value is captured, else the binding of symbols in the environment is captured. Thus these symbols' meaning remains the same wherever they are used, this is hygienicness of macro.

syntax-rules is very useful, but it also has limitations^{2, 3}, it does not provides ways to modify the templates, there exists some cases where it is harder to write with syntax-rules than define-macro. For more details about syntax-rules system, this article⁴ is very useful.

The syntax-case system is somehow similar to gensym approach, but it is an extreme. It has the same power as define-macro but retains hygenicness. Using syntax-case, the macro when can be rewritten as:

(define-syntax when
  (lambda (stx)
    (syntax-case stx ()
	((_ cond exp1 . exp2)
	 #'(if cond
	       (begin exp1 . exp2))))))

Here, the form #'(if cond (begin exp1 . exp2)) is the corresponding part of `(if ,cond (begin ,exp . ,rest)) of define-macro, they have similar structure. The difference here is only that the body of syntax-case should return a syntax object, not a mere list as define-macro returns. But in syntax-case, symbol if and begin was captured the definition of defining environment while cond, exp1 and exp2 are replaced with applying arguments when the macro is applied, since they appear in the pattern form of syntax-case. This is how hygenicness is archieved.

The macro swap!:

(define-syntax swap!
  (lambda (stx)
    (syntax-case stx ()
      ((_ op1 op2)
       #'(let ((tmp op1))
	   (set! op1 op2)
	   (set! op2 tmp))))))

The expanded result(using 'expand' of ChezScheme (expand '(swap! var1 var2))):

(let ([#{tmp cvbrbh0gnpwjvy0dl1m1nb-1} var1])
  (set! var1 var2)
  (set! var2 #{tmp cvbrbh0gnpwjvy0dl1m1nb-1}))

It is shown that hygenicness is automatically done by the same way of gensym.

The 'syntax' objects is a special type objects defined in scheme implementations which has syntax-case system ported. They can be converted to scheme objects on demand, and when returned by syntax-case, the macro expansion is done.

The following examples shows a more complex usage of syntax-case, upon expansion, the macro will be expanded to n-th fibonacci number:

;; this macro using iterative approach
(define-syntax fib
  (lambda (x)
    (syntax-case x ()
      ((fib n)
       #'(fib n 1 0))
      ((fib 0 num2 num3) (syntax->datum #'num2))
      ((fib num1 num2 num3)
       (with-syntax ((n (datum->syntax #'fib
				       (- (syntax->datum #'num1)
					  1)))
		     (n2 (datum->syntax #'fib
					(+ (syntax->datum #'num2)
					   (syntax->datum #'num3))))
		     (n3 (syntax num2)))
	 #'(fib n n2 n3))))))
;; this macro is recursive and much slower
(define-syntax fib2
  (lambda (x)
    (syntax-case x ()
      ((fib2 0)
       #'1)
      ((fib2 1)
       #'1)
      ((fib2 n)
       (with-syntax ((n2 (datum->syntax #'fib2
					(- (syntax->datum #'n) 1)))
		     (n3 (datum->syntax #'fib2
					(- (syntax->datum #'n) 2))))
	 #'(+ (fib2 n2) (fib2 n3)))))))

The syntax-case system is so powerful that even the syntax-rules system is just a derivation of it⁵.

In rencent Elixr⁶ release(1.2), a ~with~⁷ clause is introduced to provide guarded execution of some code, the same effect can be easily archieved in scheme by using macros:

(define-syntax and-let*-values-check
  (lambda (stx)
    (syntax-case stx (else)
      ((kw (?bind0 ?bind1 ...) ?body0 ... (else ?expr0 ...))
       (syntax-case #'?bind0 ()
	 (((?b0 ?b1 ...) ?exp)
	  (let* ((?check-b0 (car (generate-temporaries (list #'?b0)))))
	    #`(let-values (((#,?check-b0 ?b1 ...) ?exp))
		(if (equal? #,?check-b0 ?b0)
		    (kw (?bind1 ...) ?body0 ...)
		    (begin ?expr0 ...)))))))
      ((_ () ?body0 ... (else ?expr0 ...))
       #'(begin ?body0 ...))
      ((kw (?bind0 ...) ?body0 ...)
       #'(kw (?bind0 ...) ?body0 ... (else #f))))))

In scheme's world, the word 'syntax' mostly refers to two things, one is the lexical structure of program, which defined by the reader, usually they are s-expressions or converted to s-expressions by reader, other one is the context structure of a program, which defines how program is executed. Due to the powerfulness of s-expressions, the first meaning of syntax is insignificant compare to the second one. And with the macro system, one can easily extend the syntax scheme, just as shown in ⁵. The lastest example of previous section even shows how to borrow syntax-sugars from other languages. Yes, syntax-sugar, as its name, they are not essentials but sugars just make life happier. And even type systems can be defined using macros⁸.

This is my new blog system, which is still under construction. In this space, I will talk about Scheme, Linux, and other related topics, these will record my adventure into Schemer's land.

For integrating these things, these tools are used:

• emacs
• org-mode
• org-static-blog
• language-tool
• emacs langtool
• git

and more.