SoFunction
Updated on 2024-11-12

Golang implementation of the http server to provide compressed file download function

Recently encountered a need to download static html reports, the need to provide the form of compressed packages to complete the download function, the realization of the process found that the relevant documents are very miscellaneous, so I summarize the implementation of their own.

Development Environment:

System environment: MacOS + Chrome
Framework: beego
Compression: tar + gzip
Target zip file: self-contained data and static html files for all packages

First of all, the first to mention the realization of the http server file download, in fact, is in the back end to return to the front end of the packet, the data header is set to download the file format, so that the front end to receive the response back, it will be directly triggered by the download function (as usual when we click in the chrome to download that)
The data header is set in the following format:

func (c *Controller)Download() {
  //... Logic for generating file information
  //
  //rw for responseWriter
  rw := 
  //Specify the name of the downloaded file
  ().Set("Content-Disposition", "attachment; filename="+"(File name)")
  ().Set("Content-Description", "File Transfer")
  //Indicate the type of file to be transferred
  // In case of other types, please refer to: /http/
  ().Set("Content-Type", "application/octet-stream")
  ().Set("Content-Transfer-Encoding", "binary")
  ().Set("Expires", "0")
  ().Set("Cache-Control", "must-revalidate")
  ().Set("Pragma", "public")
  ()
  // Files are transferred using the byte slice type, in this example: b is one, then you need to call ()
  (rw, , "(File name)", (), (()))
}

In this way, the beego backend sends the packet marked as a download file in the header to the frontend, which receives it and automatically starts the download function.

However, this is only the last step in the situation, how can we compress our file before sending it to the front-end to offer it for download?

If you need to download more than one file, you need to pack it with tar and compress it with gzip, which is implemented as follows:

  // The innermost level is used for file storage.
  var b 
  // Nested tar package writer and gzip package writer
  gw := (&b)
  tw := (gw)


  dataFile := //... File generation logic, dataFile is of type File
  info, _ := ()
  header, _ := (info, "")
  //Path setting of the current file after download
   = "report" + "/" + 
  err := (header)
  if err != nil {
    (())
    return
  }
  _, err = (tw, dataFile)
  if err != nil {
    (())
  }
  //... You can continue to add files
  //tar writer and gzip writer must not be closed in the reverse order.
  ()
  ()

With the final and intermediate steps complete, we are left with the File file generation logic. Since it is a static html file, we need to write all the html references to the dependency packages all complete under the <script> and <style> tags in the generated file. In addition, in this example, the report section requires some static json data to populate the tables and images, which is stored in memory as a map. Of course it can be saved as a file before packing and compressing in the above step, but this will create concurrency problems, so we need to write all the dependency package files and data into one first, and finally convert this back to File format.

There are no well-written functions to convert files available in Golang, so we need to implement them ourselves.

The realization is as follows:

type myFileInfo struct {
  name string
  data []byte
}

func (mif myFileInfo) Name() string    { return  }
func (mif myFileInfo) Size() int64    { return int64(len()) }
func (mif myFileInfo) Mode()  { return 0444 }    // Read for all
func (mif myFileInfo) ModTime()  { return {} } // Return whatever you want
func (mif myFileInfo) IsDir() bool    { return false }
func (mif myFileInfo) Sys() interface{}  { return nil }

type MyFile struct {
  *
  mif myFileInfo
}

func (mf *MyFile) Close() error { return nil } // Noop, nothing to do

func (mf *MyFile) Readdir(count int) ([], error) {
  return nil, nil // We are not a directory but a single file
}

func (mf *MyFile) Stat() (, error) {
  return , nil
}

Dependent packet and data write logic:

func testWrite(data map[string]interface{}, taskId string)  {
  // final html generated, open html template
  tempfileP, _ := ("views/traffic/")
  info, _ := ()
  html := make([]byte, ())
  _, err := (html)
  // Write data to html
  var b 
  // Create a Json encoder
  encoder := (&b)

  err = (data)
  if err != nil {
    (())
  }
  
  // Add the json data to the html template.
  // This is done by inserting a special replacement field in the html template, in this case {Data_Json_Source}.
  html = (html, []byte("{Data_Json_Source}"), (), 1)

  // Add static files to html
  // If it's .css, add <style></style> tags before and after.
  // If it's .js, add <script> <script> tags before and after.
  allStaticFiles := make([][]byte, 0)
  // jquery needs to be added first
  tempfilename := "static/report/"

  tempfileP, _ = (tempfilename)
  info, _ = (tempfilename)
  curFileByte := make([]byte, ())
  _, err = (curFileByte)

  allStaticFiles = append(allStaticFiles, []byte("<script>"))
  allStaticFiles = append(allStaticFiles, curFileByte)
  allStaticFiles = append(allStaticFiles, []byte("</script>"))
  // All remaining static files
  staticFiles, _ := ("static/report/")
  for _, tempfile := range staticFiles {
    if () == "" {
      continue
    }
    tempfilename := "static/report/" + ()

    tempfileP, _ := (tempfilename)
    info, _ := (tempfilename)
    curFileByte := make([]byte, ())
    _, err := (curFileByte)
    if err != nil {
      (())
    }
    if isJs, _ := (`\.js$`, tempfilename); isJs {
      allStaticFiles = append(allStaticFiles, []byte("<script>"))
      allStaticFiles = append(allStaticFiles, curFileByte)
      allStaticFiles = append(allStaticFiles, []byte("</script>"))
    } else if isCss, _ := (`\.css$`, tempfilename); isCss {
      allStaticFiles = append(allStaticFiles, []byte("<style>"))
      allStaticFiles = append(allStaticFiles, curFileByte)
      allStaticFiles = append(allStaticFiles, []byte("</style>"))
    }
    ()
  }
  
  // Convert to format for return
  mf := &MyFile{
    Reader: (html),
    mif: myFileInfo{
      name: "",
      data: html,
    },
  }
  var f  = mf
  return f
}

OK! So far, the back-end file generation - & gt; packing - & gt; compression have been done, we string them together:

func (c *Controller)Download() {
  var b 
  gw := (&b)

  tw := (gw)

  // Generate a dynamic report and add it to the package.
  // Call the testWrite method from above.
  dataFile := testWrite(responseByRules, strTaskId)
  info, _ := ()
  header, _ := (info, "")
   = "report_" + strTaskId + "/" + 
  err := (header)
  if err != nil {
    (())
    return
  }
  _, err = (tw, dataFile)
  if err != nil {
    (())
  }

  ()
  ()
  rw := 
  ().Set("Content-Disposition", "attachment; filename="+"report_"+strTaskId+".")
  ().Set("Content-Description", "File Transfer")
  ().Set("Content-Type", "application/octet-stream")
  ().Set("Content-Transfer-Encoding", "binary")
  ().Set("Expires", "0")
  ().Set("Cache-Control", "must-revalidate")
  ().Set("Pragma", "public")
  ()
  (rw, , "report_"+strTaskId+".", (), (()))
}

The back-end part has been fully realized, the front-end part of how to receive it, in this case I made a button nested <a> tags to request:.

<a href="/traffic/download_indicator?task_id={{$.taskId}}&task_type={{$.taskType}}&status={{$.status}}&agent_addr={{$.agentAddr}}&glaucus_addr={{$.glaucusAddr}}" rel="external nofollow" >
   <button style="font-family: 'SimHei';font-size: 14px;font-weight: bold;color: #0d6aad;text-decoration: underline;margin-left: 40px;" type="button" class="btn btn-link">Download Reports</button>
</a>

In this way, after clicking on the Download Report button in the front-end page, the download will be automatically initiated to download the file passed back from our back-end.

to this article on the Golang http server to provide compressed file download function of the article is introduced to this, more related to Golang http server compressed download content please search for my previous articles or continue to browse the following related articles I hope that you will support me in the future more!