{"id":225,"date":"2019-08-22T12:46:37","date_gmt":"2019-08-22T12:46:37","guid":{"rendered":"https:\/\/golangbyexamples.com\/?p=225"},"modified":"2019-11-13T15:56:59","modified_gmt":"2019-11-13T15:56:59","slug":"protocol-buffers-go","status":"publish","type":"post","link":"https:\/\/golangbyexamples.com\/protocol-buffers-go\/","title":{"rendered":"Protocol Buffers and Go: Getting started"},"content":{"rendered":"\n<p class=\"has-medium-font-size\">In this tutorial, we will see how protocol buffers can be used in the context of GO Language.<\/p>\n\n\n\n<p class=\"has-medium-font-size\"><strong>What is Protocol Buffer<\/strong><\/p>\n\n\n\n<p class=\"has-medium-font-size\">Protocol Buffers are data format which stores data in a structured format. Data in the protocol buffer format can be serialized and deserialized by multiple languages.<\/p>\n\n\n\n<p class=\"has-medium-font-size\">Sounds Confusing. You can think it on lines of JSON, XML but it has loads of advantages to offer. Still confusing? then don&#8217;t worry as we go along the tutorial we will understand why even a new data format is even needed.<\/p>\n\n\n\n<p class=\"has-medium-font-size\">Let&#8217;s see first with an example of the simplest protocol buffer file.<\/p>\n\n\n\n<p><strong>person.proto<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-go\">syntax = \"proto3\";\n\nmessage Person {\n    string name = 1;\n}<\/code><\/pre>\n\n\n\n<p class=\"has-medium-font-size\">A couple of points to note about the above file.<\/p>\n\n\n\n<ul class=\"has-medium-font-size wp-block-list\"><li>This file is just a blueprint of how our Protocol Buffer structure is. There is no data associated yet. This is different from JSON\/XML , in which file also represents the actual data.<\/li><li>In the above file there is a <strong>person <\/strong>message with a field <strong>name <\/strong>of type <strong>string.\u00a0 &#8220;proto3&#8221;<\/strong>\u00a0means that the message written is incompatible with Protocol Buffer version three.\u00a0\u00a0<\/li><li>From the above example, there is one difference you can notice from JSON i.e, it has type information. This type of information will be useful in the auto-generation of code in different languages. Let&#8217;s see an example of\u00a0 auto-generation in Golang<\/li><\/ul>\n\n\n\n<p class=\"has-medium-font-size\"><strong>Auto Generation of GO Code:<\/strong><\/p>\n\n\n\n<ul class=\"has-medium-font-size wp-block-list\"><li>We can generate a corresponding Golang code using the above person.proto file. But to do that we need to do some installations:<\/li><\/ul>\n\n\n\n<p class=\"has-medium-font-size\"><strong>Installations:<\/strong><\/p>\n\n\n\n<ul class=\"has-medium-font-size wp-block-list\"><li>First install the C++ implementation of Protocol Buffers. Each platform has its own way of installation. See this link &#8211;&nbsp;<a href=\"https:\/\/github.com\/protocolbuffers\/protobuf\/blob\/master\/src\/README.md\">https:\/\/github.com\/protocolbuffers\/protobuf\/blob\/master\/src\/README.md<\/a><\/li><li>Install Golang<\/li><li>Install protoc-gen-go&nbsp;&#8211;&nbsp;<strong>go get -u <\/strong><a href=\"http:\/\/github.com\/golang\/protobuf\/protoc-gen-go\">github.com\/golang\/protobuf\/protoc-gen-go<\/a><strong>&nbsp;. <\/strong>This package will be used for auto generation of Go code<\/li><\/ul>\n\n\n\n<p class=\"has-medium-font-size\">After the installation is done then cd to the directory which contains the <strong>person.proto <\/strong>file. Run this command:<\/p>\n\n\n\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-bash\">protoc -I .\/ --go_out=.\/ .\/person.proto<\/code><\/pre>\n\n\n\n<p class=\"has-medium-font-size\">It will generate a data access go file with name&nbsp;<strong>person.pb.go&nbsp;<\/strong>&nbsp;in the same directory.&nbsp;<\/p>\n\n\n\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-go\">\/\/ Code generated by protoc-gen-go. DO NOT EDIT.\n\/\/ source: person.proto\n\n\npackage person\n\n\nimport (\n    fmt \"fmt\"\n    proto \"github.com\/golang\/protobuf\/proto\"\n    math \"math\"\n)\n\n\n\/\/ Reference imports to suppress errors if they are not otherwise used.\nvar _ = proto.Marshal\nvar _ = fmt.Errorf\nvar _ = math.Inf\n\n\n\n\n\/\/ This is a compile-time assertion to ensure that this generated file\n\/\/ is compatible with the proto package it is being compiled against.\n\/\/ A compilation error at this line likely means your copy of the\n\/\/ proto package needs to be updated.\nconst _ = proto.ProtoPackageIsVersion3 \/\/ please upgrade the proto package\n\n\ntype Person struct {\n    Name                 string   `protobuf:\"bytes,1,opt,name=name,proto3\" json:\"name,omitempty\"`\n    XXX_NoUnkeyedLiteral struct{} `json:\"-\"`\n    XXX_unrecognized     []byte   `json:\"-\"`\n    XXX_sizecache        int32    `json:\"-\"`\n}\n\n\n\n\nfunc (m *Person) Reset()         { *m = Person{} }\nfunc (m *Person) String() string { return proto.CompactTextString(m) }\nfunc (*Person) ProtoMessage()    {}\nfunc (*Person) Descriptor() ([]byte, []int) {\n    return fileDescriptor_4c9e10cf24b1156d, []int{0}\n}\n\n\nfunc (m *Person) XXX_Unmarshal(b []byte) error {\n    return xxx_messageInfo_Person.Unmarshal(m, b)\n}\nfunc (m *Person) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {\n    return xxx_messageInfo_Person.Marshal(b, m, deterministic)\n}\nfunc (m *Person) XXX_Merge(src proto.Message) {\n    xxx_messageInfo_Person.Merge(m, src)\n}\nfunc (m *Person) XXX_Size() int {\n    return xxx_messageInfo_Person.Size(m)\n}\nfunc (m *Person) XXX_DiscardUnknown() {\n    xxx_messageInfo_Person.DiscardUnknown(m)\n}\n\n\nvar xxx_messageInfo_Person proto.InternalMessageInfo\n\n\nfunc (m *Person) GetName() string {\n    if m != nil {\n        return m.Name\n    }\n    return \"\"\n}\n\n\nfunc init() {\n    proto.RegisterType((*Person)(nil), \"Person\")\n}\n\n\nfunc init() { proto.RegisterFile(\"person.proto\", fileDescriptor_4c9e10cf24b1156d) }\n\n\nvar fileDescriptor_4c9e10cf24b1156d = []byte{\n    \/\/ 67 bytes of a gzipped FileDescriptorProto\n    0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x29, 0x48, 0x2d, 0x2a,\n    0xce, 0xcf, 0xd3, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x57, 0x92, 0xe1, 0x62, 0x0b, 0x00, 0xf3, 0x85,\n    0x84, 0xb8, 0x58, 0xf2, 0x12, 0x73, 0x53, 0x25, 0x18, 0x15, 0x18, 0x35, 0x38, 0x83, 0xc0, 0xec,\n    0x24, 0x36, 0xb0, 0x22, 0x63, 0x40, 0x00, 0x00, 0x00, 0xff, 0xff, 0xa4, 0xaf, 0x53, 0x72, 0x34,\n    0x00, 0x00, 0x00,\n}<\/code><\/pre>\n\n\n\n<p class=\"has-medium-font-size\">Now the biggest question is what is this person.pb.go file which has been auto-generated by protoc using person.proto . The first couple of points to notice<\/p>\n\n\n\n<ul class=\"has-medium-font-size wp-block-list\"><li><strong>Person <\/strong>struct like below. See how type information of person.proto file is used to know what type of field <strong>Name\u00a0<\/strong>is a string<\/li><\/ul>\n\n\n\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-go\">type Person struct {\n    Name                 string   `protobuf:\"bytes,1,opt,name=name,proto3\" json:\"name,omitempty\"`\n    XXX_NoUnkeyedLiteral struct{} `json:\"-\"`\n    XXX_unrecognized     []byte   `json:\"-\"`\n    XXX_sizecache        int32    `json:\"-\"`\n}<\/code><\/pre>\n\n\n\n<ul class=\"has-medium-font-size wp-block-list\"><li>\u00a0\u00a0<b>Person <\/b>struct also implements a couple of methods that make if of interface type\u00a0<strong>proto.Message.<\/strong><\/li><\/ul>\n\n\n\n<p class=\"has-medium-font-size\">So basically this autogenerated file generates data accessors for Person struct and it provides methods that allow marshaling\/unmarshalling of Person struct type to\/from actual bytes. Now let&#8217;s write a main.go program to actually create concrete objects of <strong>Person<\/strong> struct. Here we will see a couple of advantages which Protocol Buffer has to offer. The below program also shows the read and write of Person struct to file.<\/p>\n\n\n\n<p class=\"has-medium-font-size\"><strong>main.go<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-go\">package main\n\n\nimport (\n    \"fmt\"\n    \"io\/ioutil\"\n    \"log\"\n    proto \"github.com\/golang\/protobuf\/proto\"\n)\n\n\nfunc main() {\n    person := &Person{Name: \"XXX\"}\n    fmt.Printf(\"Person's name is %s\\n\", person.GetName())\n\n\n\n\n    \/\/Now lets write this person object to file\n    out, err := proto.Marshal(person)\n    if err != nil {\n        log.Fatalf(\"Serialization error: %s\", err.Error())\n    }\n    if err := ioutil.WriteFile(\"person.bin\", out, 0644); err != nil {\n        log.Fatalf(\"Write File Error: %s \", err.Error())\n    }\n    fmt.Println(\"Write Success\")\n\n\n\n\n    \/\/Read from file\n    in, err := ioutil.ReadFile(\"person.bin\")\n    if err != nil {\n        log.Fatalf(\"Read File Error: %s \", err.Error())\n    }\n    person2 := &Person{}\n    err2 := proto.Unmarshal(in, person2)\n    if err2 != nil {\n        log.Fatalf(\"DeSerialization error: %s\", err.Error())\n    }\n\n\n    fmt.Println(\"Read Success\")\n    fmt.Printf(\"Person2's name is %s\\n\", person2.GetName())\n}<\/code><\/pre>\n\n\n\n<p class=\"has-medium-font-size\">To run this file first install\u00a0protobuf\/proto using &#8220;go get\u00a0<a href=\"http:\/\/github.com\/golang\/protobuf\/prot\">github.com\/golang\/protobuf\/prot<\/a>o&#8221; and then run this file using the command &#8220;go run *.go&#8221;<strong>Output:<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-\">Person's name is XXX\nWrite Success\nRead Success\nPerson2's name is XXX<\/code><\/pre>\n\n\n\n<p class=\"has-medium-font-size\">Notice that in the above program we<\/p>\n\n\n\n<ul class=\"has-medium-font-size wp-block-list\"><li>We write a concrete <strong>person <\/strong>struct to a file <strong>&#8220;person.bin&#8221;. <\/strong>It is a binary file and not human readable.<\/li><li>Also we read from the file. It is able to successfully read and print &#8220;<strong>Person2&#8217;s name is XXX&#8221;<\/strong><\/li><\/ul>\n\n\n\n<p class=\"has-medium-font-size\">The astonishing thing about\u00a0the <strong>&#8220;person.bin&#8221; file<\/strong>\u00a0is that it is of only 5 bytes as compared if you create a JSON file with the same data that will be of size more than 15 bytes.\u00a0\u00a0Also Marshal and UnMarshal from bytes to concrete objects and vice versa are also very fast as compared to unmarshal and marshaling of JSON files.\u00a0<br>Now we have provided the theory. Let&#8217;s write once again the advantages of using Protocol Buffers<\/p>\n\n\n\n<ol class=\"has-medium-font-size wp-block-list\"><li>More clear and less ambiguous than corresponding JSON and XML as there is also type information stored with them.<\/li><li>The data stored is relatively smaller in size of almost 2- 3 times smaller.<\/li><li>It is much faster. For example, serialization and deserialization is faster with protocol Buffers<\/li><li>Automated Code Generation &#8211; You write a protocol buffer file and you automatically get a corresponding GO file generated<\/li><li>A protocol buffer is used in GRPC which is next-generation replacement of REST protocol &#8211; Watch space for here, we will add an article on it soon.<\/li><\/ol>\n\n\n\n<p class=\"has-medium-font-size\"><strong>Conclusion: <\/strong>Protocol Buffers have much to offer other than what we have discussed in the article. This provides a quick overview of what protocol buffers are and what are their advantages as compared to JSON\/XML format.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this tutorial, we will see how protocol buffers can be used in the context of GO Language. What is Protocol Buffer Protocol Buffers are data format which stores data in a&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false},"categories":[1],"tags":[19,3,21,18,20],"class_list":["post-225","post","type-post","status-publish","format-standard","hentry","category-tech","tag-buffer","tag-go","tag-grpc","tag-protocol","tag-protocolbuffer"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.7 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Protocol Buffers and Go: Getting started - Welcome To Golang By Example<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/golangbyexamples.com\/protocol-buffers-go\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Protocol Buffers and Go: Getting started - Welcome To Golang By Example\" \/>\n<meta property=\"og:description\" content=\"In this tutorial, we will see how protocol buffers can be used in the context of GO Language. What is Protocol Buffer Protocol Buffers are data format which stores data in a...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/golangbyexamples.com\/protocol-buffers-go\/\" \/>\n<meta property=\"og:site_name\" content=\"Welcome To Golang By Example\" \/>\n<meta property=\"article:published_time\" content=\"2019-08-22T12:46:37+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-11-13T15:56:59+00:00\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/golangbyexamples.com\/protocol-buffers-go\/\",\"url\":\"https:\/\/golangbyexamples.com\/protocol-buffers-go\/\",\"name\":\"Protocol Buffers and Go: Getting started - Welcome To Golang By Example\",\"isPartOf\":{\"@id\":\"https:\/\/golangbyexamples.com\/#website\"},\"datePublished\":\"2019-08-22T12:46:37+00:00\",\"dateModified\":\"2019-11-13T15:56:59+00:00\",\"author\":{\"@id\":\"https:\/\/golangbyexamples.com\/#\/schema\/person\/8833ea7638dafd763cb1db6c0ca4576f\"},\"breadcrumb\":{\"@id\":\"https:\/\/golangbyexamples.com\/protocol-buffers-go\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/golangbyexamples.com\/protocol-buffers-go\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/golangbyexamples.com\/protocol-buffers-go\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/golangbyexamples.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Protocol Buffers and Go: Getting started\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/golangbyexamples.com\/#website\",\"url\":\"https:\/\/golangbyexamples.com\/\",\"name\":\"Welcome To Golang By Example\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/golangbyexamples.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/golangbyexamples.com\/#\/schema\/person\/8833ea7638dafd763cb1db6c0ca4576f\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/golangbyexamples.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g\",\"caption\":\"admin\"},\"url\":\"https:\/\/golangbyexamples.com\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Protocol Buffers and Go: Getting started - Welcome To Golang By Example","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/golangbyexamples.com\/protocol-buffers-go\/","og_locale":"en_US","og_type":"article","og_title":"Protocol Buffers and Go: Getting started - Welcome To Golang By Example","og_description":"In this tutorial, we will see how protocol buffers can be used in the context of GO Language. What is Protocol Buffer Protocol Buffers are data format which stores data in a...","og_url":"https:\/\/golangbyexamples.com\/protocol-buffers-go\/","og_site_name":"Welcome To Golang By Example","article_published_time":"2019-08-22T12:46:37+00:00","article_modified_time":"2019-11-13T15:56:59+00:00","author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/golangbyexamples.com\/protocol-buffers-go\/","url":"https:\/\/golangbyexamples.com\/protocol-buffers-go\/","name":"Protocol Buffers and Go: Getting started - Welcome To Golang By Example","isPartOf":{"@id":"https:\/\/golangbyexamples.com\/#website"},"datePublished":"2019-08-22T12:46:37+00:00","dateModified":"2019-11-13T15:56:59+00:00","author":{"@id":"https:\/\/golangbyexamples.com\/#\/schema\/person\/8833ea7638dafd763cb1db6c0ca4576f"},"breadcrumb":{"@id":"https:\/\/golangbyexamples.com\/protocol-buffers-go\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/golangbyexamples.com\/protocol-buffers-go\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/golangbyexamples.com\/protocol-buffers-go\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/golangbyexamples.com\/"},{"@type":"ListItem","position":2,"name":"Protocol Buffers and Go: Getting started"}]},{"@type":"WebSite","@id":"https:\/\/golangbyexamples.com\/#website","url":"https:\/\/golangbyexamples.com\/","name":"Welcome To Golang By Example","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/golangbyexamples.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/golangbyexamples.com\/#\/schema\/person\/8833ea7638dafd763cb1db6c0ca4576f","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/golangbyexamples.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","caption":"admin"},"url":"https:\/\/golangbyexamples.com\/author\/admin\/"}]}},"jetpack_featured_media_url":"","jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/paOs1b-3D","amp_validity":null,"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/golangbyexamples.com\/wp-json\/wp\/v2\/posts\/225","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/golangbyexamples.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/golangbyexamples.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/golangbyexamples.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/golangbyexamples.com\/wp-json\/wp\/v2\/comments?post=225"}],"version-history":[{"count":10,"href":"https:\/\/golangbyexamples.com\/wp-json\/wp\/v2\/posts\/225\/revisions"}],"predecessor-version":[{"id":522,"href":"https:\/\/golangbyexamples.com\/wp-json\/wp\/v2\/posts\/225\/revisions\/522"}],"wp:attachment":[{"href":"https:\/\/golangbyexamples.com\/wp-json\/wp\/v2\/media?parent=225"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/golangbyexamples.com\/wp-json\/wp\/v2\/categories?post=225"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/golangbyexamples.com\/wp-json\/wp\/v2\/tags?post=225"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}