Vendor temporarily

This commit is contained in:
Paul Tyng committed 2020-10-14 11:34:14 -04:00
1 parent dd8e532d18
commit 1c02ed9115
4708 files changed
+1226392 -12

No files matched your search

+7
View File
@@ -0,0 +1,7 @@
.idea/
node_modules/
*.iml
*.exe
.cover/
vendor/
cover.out
+18
View File
@@ -0,0 +1,18 @@
go_import_path: github.com/ory/dockertest/v3
language: go
services:
- docker
go:
- 1.13
before_install:
- go get github.com/mattn/goveralls github.com/sonatype-nexus-community/nancy
script:
- go list -m all | nancy -quiet
- go vet -x .
- go test -covermode=atomic -coverprofile="cover.out" .
- goveralls -coverprofile="cover.out"
+119
View File
@@ -0,0 +1,119 @@
# Contributing to ORY Dockertest
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
- [Introduction](#introduction)
- [Contributing Code](#contributing-code)
- [Disclosing vulnerabilities](#disclosing-vulnerabilities)
- [Code Style](#code-style)
- [Pull request procedure](#pull-request-procedure)
- [Communication](#communication)
- [Conduct](#conduct)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
## Introduction
Please note: We take ORY Dockertest's security and our users' trust very
seriously. If you believe you have found a security issue in ORY Dockertest,
please responsibly disclose by contacting us at hi@ory.sh.
First: if you're unsure or afraid of anything, just ask or submit the issue or
pull request anyways. You won't be yelled at for giving it your best effort. The
worst that can happen is that you'll be politely asked to change something. We
appreciate any sort of contributions, and don't want a wall of rules to get in
the way of that.
That said, if you want to ensure that a pull request is likely to be merged,
talk to us! You can find out our thoughts and ensure that your contribution
won't clash or be obviated by ORY Dockertest's normal direction. A great way to
do this is via the [ORY Community](https://community.ory.sh/) or join the
[ORY Chat](https://www.ory.sh/chat).
## Contributing Code
Unless you are fixing a known bug, we **strongly** recommend discussing it with
the core team via a GitHub issue or [in our chat](https://www.ory.sh/chat)
before getting started to ensure your work is consistent with ORY Dockertest's
roadmap and architecture.
All contributions are made via pull request. Note that **all patches from all
contributors get reviewed**. After a pull request is made other contributors
will offer feedback, and if the patch passes review a maintainer will accept it
with a comment. When pull requests fail testing, authors are expected to update
their pull requests to address the failures until the tests pass and the pull
request merges successfully.
At least one review from a maintainer is required for all patches (even patches
from maintainers).
Reviewers should leave a "LGTM" comment once they are satisfied with the patch.
If the patch was submitted by a maintainer with write access, the pull request
should be merged by the submitter after review.
## Disclosing vulnerabilities
Please disclose vulnerabilities exclusively to [hi@ory.am](mailto:hi@ory.am). Do
not use GitHub issues.
## Code Style
Please follow these guidelines when formatting source code:
- Go code should match the output of `gofmt -s`
## Pull request procedure
To make a pull request, you will need a GitHub account; if you are unclear on
this process, see GitHub's documentation on
[forking](https://help.github.com/articles/fork-a-repo) and
[pull requests](https://help.github.com/articles/using-pull-requests). Pull
requests should be targeted at the `master` branch. Before creating a pull
request, go through this checklist:
1. Create a feature branch off of `master` so that changes do not get mixed up.
1. [Rebase](http://git-scm.com/book/en/Git-Branching-Rebasing) your local
changes against the `master` branch.
1. Run the full project test suite with the `go test ./...` (or equivalent)
command and confirm that it passes.
1. Run `gofmt -s` (if the project is written in Go).
1. Ensure that each commit has a subsystem prefix (ex: `controller:`).
Pull requests will be treated as "review requests," and maintainers will give
feedback on the style and substance of the patch.
Normally, all pull requests must include tests that test your change.
Occasionally, a change will be very difficult to test for. In those cases,
please include a note in your commit message explaining why.
## Communication
We use [discord](https://www.ory.sh/chat). You are welcome to drop in and ask
questions, discuss bugs, etc.
## Conduct
Whether you are a regular contributor or a newcomer, we care about making this
community a safe place for you and we've got your back.
- We are committed to providing a friendly, safe and welcoming environment for
all, regardless of gender, sexual orientation, disability, ethnicity,
religion, or similar personal characteristic.
- Please avoid using nicknames that might detract from a friendly, safe and
welcoming environment for all.
- Be kind and courteous. There is no need to be mean or rude.
- We will exclude you from interaction if you insult, demean or harass anyone.
In particular, we do not tolerate behavior that excludes people in socially
marginalized groups.
- Private harassment is also unacceptable. No matter who you are, if you feel
you have been or are being harassed or made uncomfortable by a community
member, please contact one of the channel ops or a member of the ORY
Dockertest core team immediately.
- Likewise any spamming, trolling, flaming, baiting or other attention-stealing
behaviour is not welcome.
We welcome discussion about creating a welcoming, safe, and productive
environment for the community. If you have any questions, feedback, or concerns
[please let us know](https://www.ory.sh/chat).
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+121
View File
@@ -0,0 +1,121 @@
<h1 align="center"><img src="./docs/images/banner_dockertest.png" alt="ORY Dockertest"></h1>
[![Build Status](https://travis-ci.org/ory/dockertest.svg)](https://travis-ci.org/ory/dockertest?branch=master)
[![Coverage Status](https://coveralls.io/repos/github/ory/dockertest/badge.svg?branch=v3)](https://coveralls.io/github/ory/dockertest?branch=v3)
Use Docker to run your Go language integration tests against third party services on **Microsoft Windows, Mac OSX and Linux**!
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents**
- [Why should I use Dockertest?](#why-should-i-use-dockertest)
- [Installing and using Dockertest](#installing-and-using-dockertest)
- [Using Dockertest](#using-dockertest)
- [Examples](#examples)
- [Troubleshoot & FAQ](#troubleshoot-&-faq)
- [Out of disk space](#out-of-disk-space)
- [Removing old containers](#removing-old-containers)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
## Why should I use Dockertest?
When developing applications, it is often necessary to use services that talk to a database system.
Unit Testing these services can be cumbersome because mocking database/DBAL is strenuous. Making slight changes to the
schema implies rewriting at least some, if not all of the mocks. The same goes for API changes in the DBAL.
To avoid this, it is smarter to test these specific services against a real database that is destroyed after testing.
Docker is the perfect system for running unit tests as you can spin up containers in a few seconds and kill them when
the test completes. The Dockertest library provides easy to use commands for spinning up Docker containers and using
them for your tests.
## Installing and using Dockertest
Using Dockertest is straightforward and simple. Check the [releases tab](https://github.com/ory/dockertest/releases)
for available releases.
To install dockertest, run
```
go get -u github.com/ory/dockertest/v3
```
or
```
dep ensure -add github.com/ory/dockertest@v3.x.y
```
### Using Dockertest
```go
package dockertest_test
import (
"database/sql"
"fmt"
"log"
"os"
"testing"
_ "github.com/go-sql-driver/mysql"
"github.com/ory/dockertest/v3"
)
var db *sql.DB
func TestMain(m *testing.M) {
// uses a sensible default on windows (tcp/http) and linux/osx (socket)
pool, err := dockertest.NewPool("")
if err != nil {
log.Fatalf("Could not connect to docker: %s", err)
}
// pulls an image, creates a container based on it and runs it
resource, err := pool.Run("mysql", "5.7", []string{"MYSQL_ROOT_PASSWORD=secret"})
if err != nil {
log.Fatalf("Could not start resource: %s", err)
}
// exponential backoff-retry, because the application in the container might not be ready to accept connections yet
if err := pool.Retry(func() error {
var err error
db, err = sql.Open("mysql", fmt.Sprintf("root:secret@(localhost:%s)/mysql", resource.GetPort("3306/tcp")))
if err != nil {
return err
}
return db.Ping()
}); err != nil {
log.Fatalf("Could not connect to docker: %s", err)
}
code := m.Run()
// You can't defer this because os.Exit doesn't care for defer
if err := pool.Purge(resource); err != nil {
log.Fatalf("Could not purge resource: %s", err)
}
os.Exit(code)
}
func TestSomething(t *testing.T) {
// db.Query()
}
```
### Examples
We provide code examples for well known services in the [examples](examples/) directory, check them out!
## Troubleshoot & FAQ
### Out of disk space
Try cleaning up the images with [docker-cleanup-volumes](https://github.com/chadoe/docker-cleanup-volumes).
### Removing old containers
Sometimes container clean up fails. Check out
[this stackoverflow question](http://stackoverflow.com/questions/21398087/how-to-delete-dockers-images) on how to fix this. You may also set an absolute lifetime on containers:
```go
resource.Expire(60) // Tell docker to hard kill the container in 60 seconds
```
+28
View File
@@ -0,0 +1,28 @@
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*
- [Security Policy](#security-policy)
- [Supported Versions](#supported-versions)
- [Reporting a Vulnerability](#reporting-a-vulnerability)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
# Security Policy
## Supported Versions
We release patches for security vulnerabilities. Which versions are eligible
receiving such patches depend on the CVSS v3.0 Rating:
| CVSS v3.0 | Supported Versions |
| --------- | ----------------------------------------- |
| 9.0-10.0 | Releases within the previous three months |
| 4.0-8.9 | Most recent release |
## Reporting a Vulnerability
Please report (suspected) security vulnerabilities to
**[security@ory.sh](mailto:security@ory.sh)**. You will receive a response from
us within 48 hours. If the issue is confirmed, we will release a patch as soon
as possible depending on complexity but historically within a few days.
+192
View File
@@ -0,0 +1,192 @@
# This is the official list of go-dockerclient authors for copyright purposes.
Abhishek Chanda
Adam Bell-Hanssen
Adnan Khan
Adrien Kohlbecker
Aldrin Leal
Alex Dadgar
Alfonso Acosta
André Carvalho
Andreas Jaekle
Andrew Snodgrass
Andrews Medina
Andrey Sibiryov
Andy Goldstein
Anirudh Aithal
Antonio Murdaca
Artem Sidorenko
Arthur Rodrigues
Ben Marini
Ben McCann
Ben Parees
Benno van den Berg
Bradley Cicenas
Brendan Fosberry
Brian Lalor
Brian P. Hamachek
Brian Palmer
Bryan Boreham
Burke Libbey
Carlos Diaz-Padron
Carson A
Cássio Botaro
Cesar Wong
Cezar Sa Espinola
Changping Chen
Cheah Chu Yeow
cheneydeng
Chris Bednarski
Chris Stavropoulos
Christian Stewart
Christophe Mourette
Clayton Coleman
Clint Armstrong
CMGS
Colin Hebert
Craig Jellick
Damien Lespiau
Damon Wang
Dan Williams
Daniel, Dao Quang Minh
Daniel Garcia
Daniel Hiltgen
Daniel Nephin
Daniel Tsui
Darren Shepherd
Dave Choi
David Huie
Dawn Chen
Denis Makogon
Derek Petersen
Dinesh Subhraveti
Drew Wells
Ed
Elias G. Schneevoigt
Erez Horev
Eric Anderson
Eric J. Holmes
Eric Mountain
Erwin van Eyk
Ethan Mosbaugh
Ewout Prangsma
Fabio Rehm
Fatih Arslan
Felipe Oliveira
Flavia Missi
Florent Aide
Francisco Souza
Frank Groeneveld
George Moura
Grégoire Delattre
Guilherme Rezende
Guillermo Álvarez Fernández
Harry Zhang
He Simei
Isaac Schnitzer
Ivan Mikushin
James Bardin
James Nugent
Jamie Snell
Januar Wayong
Jari Kolehmainen
Jason Wilder
Jawher Moussa
Jean-Baptiste Dalido
Jeff Mitchell
Jeffrey Hulten
Jen Andre
Jérôme Laurens
Jim Minter
Johan Euphrosine
Johannes Scheuermann
John Hughes
Jorge Marey
Julian Einwag
Kamil Domanski
Karan Misra
Ken Herner
Kevin Lin
Kevin Xu
Kim, Hirokuni
Kostas Lekkas
Kyle Allan
Yunhee Lee
Liron Levin
Lior Yankovich
Liu Peng
Lorenz Leutgeb
Lucas Clemente
Lucas Weiblen
Lyon Hill
Mantas Matelis
Manuel Vogel
Marguerite des Trois Maisons
Mariusz Borsa
Martin Sweeney
Máximo Cuadros Ortiz
Michael Schmatz
Michal Fojtik
Mike Dillon
Mrunal Patel
Nate Jones
Nguyen Sy Thanh Son
Nicholas Van Wiggeren
Nick Ethier
niko83
Omeid Matten
Orivej Desh
Paul Bellamy
Paul Morie
Paul Weil
Peter Edge
Peter Jihoon Kim
Peter Teich
Phil Lu
Philippe Lafoucrière
Radek Simko
Rafe Colton
Raphaël Pinson
Reed Allman
RJ Catalano
Rob Miller
Robbert Klarenbeek
Robert Williamson
Roman Khlystik
Russell Haering
Salvador Gironès
Sam Rijs
Sami Wagiaalla
Samuel Archambault
Samuel Karp
Sebastian Borza
Seth Jennings
Shane Xie
Silas Sewell
Simon Eskildsen
Simon Menke
Skolos
Soulou
Sridhar Ratnakumar
Steven Jack
Summer Mousa
Sunjin Lee
Sunny
Swaroop Ramachandra
Tarsis Azevedo
Tim Schindler
Timothy St. Clair
Tobi Knaup
Tom Wilkie
Tonic
ttyh061
upccup
Victor Marmol
Vincenzo Prignano
Vlad Alexandru Ionescu
Weitao Zhou
Wiliam Souza
Ye Yin
Yosuke Otosu
Yu, Zou
Yuriy Bogdanov
+6
View File
@@ -0,0 +1,6 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
You can find the Docker license at the following link:
https://raw.githubusercontent.com/docker/docker/master/LICENSE
+22
View File
@@ -0,0 +1,22 @@
Copyright (c) 2013-2018, go-dockerclient authors
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+133
View File
@@ -0,0 +1,133 @@
# go-dockerclient
[![Travis Build Status](https://travis-ci.org/fsouza/go-dockerclient.svg?branch=master)](https://travis-ci.org/fsouza/go-dockerclient)
[![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/4m374pti06ubg2l7?svg=true)](https://ci.appveyor.com/project/fsouza/go-dockerclient)
[![GoDoc](https://img.shields.io/badge/api-Godoc-blue.svg?style=flat-square)](https://godoc.org/github.com/fsouza/go-dockerclient)
This package presents a client for the Docker remote API. It also provides
support for the extensions in the [Swarm API](https://docs.docker.com/swarm/swarm-api/).
This package also provides support for docker's network API, which is a simple
passthrough to the libnetwork remote API. Note that docker's network API is
only available in docker 1.8 and above, and only enabled in docker if
DOCKER_EXPERIMENTAL is defined during the docker build process.
For more details, check the [remote API
documentation](http://docs.docker.com/engine/reference/api/docker_remote_api/).
## Example
```go
package main
import (
"fmt"
"github.com/fsouza/go-dockerclient"
)
func main() {
endpoint := "unix:///var/run/docker.sock"
client, err := docker.NewClient(endpoint)
if err != nil {
panic(err)
}
imgs, err := client.ListImages(docker.ListImagesOptions{All: false})
if err != nil {
panic(err)
}
for _, img := range imgs {
fmt.Println("ID: ", img.ID)
fmt.Println("RepoTags: ", img.RepoTags)
fmt.Println("Created: ", img.Created)
fmt.Println("Size: ", img.Size)
fmt.Println("VirtualSize: ", img.VirtualSize)
fmt.Println("ParentId: ", img.ParentID)
}
}
```
## Using with TLS
In order to instantiate the client for a TLS-enabled daemon, you should use
NewTLSClient, passing the endpoint and path for key and certificates as
parameters.
```go
package main
import (
"fmt"
"github.com/fsouza/go-dockerclient"
)
func main() {
endpoint := "tcp://[ip]:[port]"
path := os.Getenv("DOCKER_CERT_PATH")
ca := fmt.Sprintf("%s/ca.pem", path)
cert := fmt.Sprintf("%s/cert.pem", path)
key := fmt.Sprintf("%s/key.pem", path)
client, _ := docker.NewTLSClient(endpoint, cert, key, ca)
// use client
}
```
If using [docker-machine](https://docs.docker.com/machine/), or another
application that exports environment variables `DOCKER_HOST`,
`DOCKER_TLS_VERIFY`, `DOCKER_CERT_PATH`, you can use NewClientFromEnv.
```go
package main
import (
"fmt"
"github.com/fsouza/go-dockerclient"
)
func main() {
client, _ := docker.NewClientFromEnv()
// use client
}
```
See the documentation for more details.
## Developing
All development commands can be seen in the [Makefile](Makefile).
Commited code must pass:
* [golint](https://github.com/golang/lint) (with some exceptions, see the Makefile).
* [go vet](https://golang.org/cmd/vet/)
* [gofmt](https://golang.org/cmd/gofmt)
* [go test](https://golang.org/cmd/go/#hdr-Test_packages)
Running `make test` will check all of these. If your editor does not
automatically call ``gofmt -s``, `make fmt` will format all go files in this
repository.
## Vendoring
go-dockerclient uses [dep](https://github.com/golang/dep/) for vendoring. If
you're using dep, you should be able to pick go-dockerclient releases and get
the proper dependencies.
With other vendoring tools, users might need to specify go-dockerclient's
dependencies manually.
## Using with Docker 1.9 and Go 1.4
There's a tag for using go-dockerclient with Docker 1.9 (which requires
compiling go-dockerclient with Go 1.4), the tag name is ``docker-1.9/go-1.4``.
The instructions below can be used to get a version of go-dockerclient that compiles with Go 1.4:
```
% git clone -b docker-1.9/go-1.4 https://github.com/fsouza/go-dockerclient.git $GOPATH/src/github.com/fsouza/go-dockerclient
% git clone -b v1.9.1 https://github.com/docker/docker.git $GOPATH/src/github.com/docker/docker
% go get github.com/fsouza/go-dockerclient
```
+185
View File
@@ -0,0 +1,185 @@
// Copyright 2015 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package docker
import (
"bytes"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"path"
"strings"
)
// ErrCannotParseDockercfg is the error returned by NewAuthConfigurations when the dockercfg cannot be parsed.
var ErrCannotParseDockercfg = errors.New("Failed to read authentication from dockercfg")
// AuthConfiguration represents authentication options to use in the PushImage
// method. It represents the authentication in the Docker index server.
type AuthConfiguration struct {
Username string `json:"username,omitempty"`
Password string `json:"password,omitempty"`
Email string `json:"email,omitempty"`
ServerAddress string `json:"serveraddress,omitempty"`
}
// AuthConfigurations represents authentication options to use for the
// PushImage method accommodating the new X-Registry-Config header
type AuthConfigurations struct {
Configs map[string]AuthConfiguration `json:"configs"`
}
// AuthConfigurations119 is used to serialize a set of AuthConfigurations
// for Docker API >= 1.19.
type AuthConfigurations119 map[string]AuthConfiguration
// dockerConfig represents a registry authentation configuration from the
// .dockercfg file.
type dockerConfig struct {
Auth string `json:"auth"`
Email string `json:"email"`
}
// NewAuthConfigurationsFromFile returns AuthConfigurations from a path containing JSON
// in the same format as the .dockercfg file.
func NewAuthConfigurationsFromFile(path string) (*AuthConfigurations, error) {
r, err := os.Open(path)
if err != nil {
return nil, err
}
return NewAuthConfigurations(r)
}
func cfgPaths(dockerConfigEnv string, homeEnv string) []string {
var paths []string
if dockerConfigEnv != "" {
paths = append(paths, path.Join(dockerConfigEnv, "config.json"))
}
if homeEnv != "" {
paths = append(paths, path.Join(homeEnv, ".docker", "config.json"))
paths = append(paths, path.Join(homeEnv, ".dockercfg"))
}
return paths
}
// NewAuthConfigurationsFromDockerCfg returns AuthConfigurations from
// system config files. The following files are checked in the order listed:
// - $DOCKER_CONFIG/config.json if DOCKER_CONFIG set in the environment,
// - $HOME/.docker/config.json
// - $HOME/.dockercfg
func NewAuthConfigurationsFromDockerCfg() (*AuthConfigurations, error) {
err := fmt.Errorf("No docker configuration found")
var auths *AuthConfigurations
pathsToTry := cfgPaths(os.Getenv("DOCKER_CONFIG"), os.Getenv("HOME"))
for _, path := range pathsToTry {
auths, err = NewAuthConfigurationsFromFile(path)
if err == nil {
return auths, nil
}
}
return auths, err
}
// NewAuthConfigurations returns AuthConfigurations from a JSON encoded string in the
// same format as the .dockercfg file.
func NewAuthConfigurations(r io.Reader) (*AuthConfigurations, error) {
var auth *AuthConfigurations
confs, err := parseDockerConfig(r)
if err != nil {
return nil, err
}
auth, err = authConfigs(confs)
if err != nil {
return nil, err
}
return auth, nil
}
func parseDockerConfig(r io.Reader) (map[string]dockerConfig, error) {
buf := new(bytes.Buffer)
buf.ReadFrom(r)
byteData := buf.Bytes()
confsWrapper := struct {
Auths map[string]dockerConfig `json:"auths"`
}{}
if err := json.Unmarshal(byteData, &confsWrapper); err == nil {
if len(confsWrapper.Auths) > 0 {
return confsWrapper.Auths, nil
}
}
var confs map[string]dockerConfig
if err := json.Unmarshal(byteData, &confs); err != nil {
return nil, err
}
return confs, nil
}
// authConfigs converts a dockerConfigs map to a AuthConfigurations object.
func authConfigs(confs map[string]dockerConfig) (*AuthConfigurations, error) {
c := &AuthConfigurations{
Configs: make(map[string]AuthConfiguration),
}
for reg, conf := range confs {
if conf.Auth == "" {
continue
}
data, err := base64.StdEncoding.DecodeString(conf.Auth)
if err != nil {
return nil, err
}
userpass := strings.SplitN(string(data), ":", 2)
if len(userpass) != 2 {
return nil, ErrCannotParseDockercfg
}
c.Configs[reg] = AuthConfiguration{
Email: conf.Email,
Username: userpass[0],
Password: userpass[1],
ServerAddress: reg,
}
}
return c, nil
}
// AuthStatus returns the authentication status for Docker API versions >= 1.23.
type AuthStatus struct {
Status string `json:"Status,omitempty" yaml:"Status,omitempty" toml:"Status,omitempty"`
IdentityToken string `json:"IdentityToken,omitempty" yaml:"IdentityToken,omitempty" toml:"IdentityToken,omitempty"`
}
// AuthCheck validates the given credentials. It returns nil if successful.
//
// For Docker API versions >= 1.23, the AuthStatus struct will be populated, otherwise it will be empty.`
//
// See https://goo.gl/6nsZkH for more details.
func (c *Client) AuthCheck(conf *AuthConfiguration) (AuthStatus, error) {
var authStatus AuthStatus
if conf == nil {
return authStatus, errors.New("conf is nil")
}
resp, err := c.do("POST", "/auth", doOptions{data: conf})
if err != nil {
return authStatus, err
}
defer resp.Body.Close()
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
return authStatus, err
}
if len(data) == 0 {
return authStatus, nil
}
if err := json.Unmarshal(data, &authStatus); err != nil {
return authStatus, err
}
return authStatus, nil
}
+43
View File
@@ -0,0 +1,43 @@
// Copyright 2014 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package docker
import "fmt"
// ChangeType is a type for constants indicating the type of change
// in a container
type ChangeType int
const (
// ChangeModify is the ChangeType for container modifications
ChangeModify ChangeType = iota
// ChangeAdd is the ChangeType for additions to a container
ChangeAdd
// ChangeDelete is the ChangeType for deletions from a container
ChangeDelete
)
// Change represents a change in a container.
//
// See https://goo.gl/Wo0JJp for more details.
type Change struct {
Path string
Kind ChangeType
}
func (change *Change) String() string {
var kind string
switch change.Kind {
case ChangeModify:
kind = "C"
case ChangeAdd:
kind = "A"
case ChangeDelete:
kind = "D"
}
return fmt.Sprintf("%s %s", kind, change.Path)
}
File diff suppressed because it is too large. Load diff
+32
View File
@@ -0,0 +1,32 @@
// Copyright 2016 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !windows
package docker
import (
"context"
"net"
"net/http"
)
// initializeNativeClient initializes the native Unix domain socket client on
// Unix-style operating systems
func (c *Client) initializeNativeClient(trFunc func() *http.Transport) {
if c.endpointURL.Scheme != unixProtocol {
return
}
sockPath := c.endpointURL.Path
tr := trFunc()
tr.Dial = func(network, addr string) (net.Conn, error) {
return c.Dialer.Dial(unixProtocol, sockPath)
}
tr.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {
return c.Dialer.Dial(unixProtocol, sockPath)
}
c.HTTPClient.Transport = tr
}
+45
View File
@@ -0,0 +1,45 @@
// Copyright 2016 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build windows
package docker
import (
"context"
"net"
"net/http"
"time"
"github.com/Microsoft/go-winio"
)
const namedPipeConnectTimeout = 2 * time.Second
type pipeDialer struct {
dialFunc func(network, addr string) (net.Conn, error)
}
func (p pipeDialer) Dial(network, address string) (net.Conn, error) {
return p.dialFunc(network, address)
}
// initializeNativeClient initializes the native Named Pipe client for Windows
func (c *Client) initializeNativeClient(trFunc func() *http.Transport) {
if c.endpointURL.Scheme != namedPipeProtocol {
return
}
namedPipePath := c.endpointURL.Path
dialFunc := func(network, addr string) (net.Conn, error) {
timeout := namedPipeConnectTimeout
return winio.DialPipe(namedPipePath, &timeout)
}
tr := trFunc()
tr.Dial = dialFunc
tr.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {
return dialFunc(network, addr)
}
c.Dialer = &pipeDialer{dialFunc}
c.HTTPClient.Transport = tr
}
File diff suppressed because it is too large. Load diff
+26
View File
@@ -0,0 +1,26 @@
// Copyright 2017 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package docker
import (
"encoding/json"
"github.com/ory/dockertest/v3/docker/types/registry"
)
// InspectDistribution returns image digest and platform information by contacting the registry
func (c *Client) InspectDistribution(name string) (*registry.DistributionInspect, error) {
path := "/distribution/" + name + "/json"
resp, err := c.do("GET", path, doOptions{})
if err != nil {
return nil, err
}
defer resp.Body.Close()
var distributionInspect registry.DistributionInspect
if err := json.NewDecoder(resp.Body).Decode(&distributionInspect); err != nil {
return nil, err
}
return &distributionInspect, nil
}
+172
View File
@@ -0,0 +1,172 @@
// Copyright 2014 Docker authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the DOCKER-LICENSE file.
package docker
import (
"encoding/json"
"fmt"
"io"
"strconv"
"strings"
)
// Env represents a list of key-pair represented in the form KEY=VALUE.
type Env []string
// Get returns the string value of the given key.
func (env *Env) Get(key string) (value string) {
return env.Map()[key]
}
// Exists checks whether the given key is defined in the internal Env
// representation.
func (env *Env) Exists(key string) bool {
_, exists := env.Map()[key]
return exists
}
// GetBool returns a boolean representation of the given key. The key is false
// whenever its value if 0, no, false, none or an empty string. Any other value
// will be interpreted as true.
func (env *Env) GetBool(key string) (value bool) {
s := strings.ToLower(strings.Trim(env.Get(key), " \t"))
if s == "" || s == "0" || s == "no" || s == "false" || s == "none" {
return false
}
return true
}
// SetBool defines a boolean value to the given key.
func (env *Env) SetBool(key string, value bool) {
if value {
env.Set(key, "1")
} else {
env.Set(key, "0")
}
}
// GetInt returns the value of the provided key, converted to int.
//
// It the value cannot be represented as an integer, it returns -1.
func (env *Env) GetInt(key string) int {
return int(env.GetInt64(key))
}
// SetInt defines an integer value to the given key.
func (env *Env) SetInt(key string, value int) {
env.Set(key, strconv.Itoa(value))
}
// GetInt64 returns the value of the provided key, converted to int64.
//
// It the value cannot be represented as an integer, it returns -1.
func (env *Env) GetInt64(key string) int64 {
s := strings.Trim(env.Get(key), " \t")
val, err := strconv.ParseInt(s, 10, 64)
if err != nil {
return -1
}
return val
}
// SetInt64 defines an integer (64-bit wide) value to the given key.
func (env *Env) SetInt64(key string, value int64) {
env.Set(key, strconv.FormatInt(value, 10))
}
// GetJSON unmarshals the value of the provided key in the provided iface.
//
// iface is a value that can be provided to the json.Unmarshal function.
func (env *Env) GetJSON(key string, iface interface{}) error {
sval := env.Get(key)
if sval == "" {
return nil
}
return json.Unmarshal([]byte(sval), iface)
}
// SetJSON marshals the given value to JSON format and stores it using the
// provided key.
func (env *Env) SetJSON(key string, value interface{}) error {
sval, err := json.Marshal(value)
if err != nil {
return err
}
env.Set(key, string(sval))
return nil
}
// GetList returns a list of strings matching the provided key. It handles the
// list as a JSON representation of a list of strings.
//
// If the given key matches to a single string, it will return a list
// containing only the value that matches the key.
func (env *Env) GetList(key string) []string {
sval := env.Get(key)
if sval == "" {
return nil
}
var l []string
if err := json.Unmarshal([]byte(sval), &l); err != nil {
l = append(l, sval)
}
return l
}
// SetList stores the given list in the provided key, after serializing it to
// JSON format.
func (env *Env) SetList(key string, value []string) error {
return env.SetJSON(key, value)
}
// Set defines the value of a key to the given string.
func (env *Env) Set(key, value string) {
*env = append(*env, key+"="+value)
}
// Decode decodes `src` as a json dictionary, and adds each decoded key-value
// pair to the environment.
//
// If `src` cannot be decoded as a json dictionary, an error is returned.
func (env *Env) Decode(src io.Reader) error {
m := make(map[string]interface{})
if err := json.NewDecoder(src).Decode(&m); err != nil {
return err
}
for k, v := range m {
env.SetAuto(k, v)
}
return nil
}
// SetAuto will try to define the Set* method to call based on the given value.
func (env *Env) SetAuto(key string, value interface{}) {
if fval, ok := value.(float64); ok {
env.SetInt64(key, int64(fval))
} else if sval, ok := value.(string); ok {
env.Set(key, sval)
} else if val, err := json.Marshal(value); err == nil {
env.Set(key, string(val))
} else {
env.Set(key, fmt.Sprintf("%v", value))
}
}
// Map returns the map representation of the env.
func (env *Env) Map() map[string]string {
if len(*env) == 0 {
return nil
}
m := make(map[string]string)
for _, kv := range *env {
parts := strings.SplitN(kv, "=", 2)
if len(parts) == 1 {
m[parts[0]] = ""
} else {
m[parts[0]] = parts[1]
}
}
return m
}
+410
View File
@@ -0,0 +1,410 @@
// Copyright 2014 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package docker
import (
"encoding/json"
"errors"
"fmt"
"io"
"math"
"net"
"net/http"
"net/http/httputil"
"sync"
"sync/atomic"
"time"
)
// APIEvents represents events coming from the Docker API
// The fields in the Docker API changed in API version 1.22, and
// events for more than images and containers are now fired off.
// To maintain forward and backward compatibility, go-dockerclient
// replicates the event in both the new and old format as faithfully as possible.
//
// For events that only exist in 1.22 in later, `Status` is filled in as
// `"Type:Action"` instead of just `Action` to allow for older clients to
// differentiate and not break if they rely on the pre-1.22 Status types.
//
// The transformEvent method can be consulted for more information about how
// events are translated from new/old API formats
type APIEvents struct {
// New API Fields in 1.22
Action string `json:"action,omitempty"`
Type string `json:"type,omitempty"`
Actor APIActor `json:"actor,omitempty"`
// Old API fields for < 1.22
Status string `json:"status,omitempty"`
ID string `json:"id,omitempty"`
From string `json:"from,omitempty"`
// Fields in both
Time int64 `json:"time,omitempty"`
TimeNano int64 `json:"timeNano,omitempty"`
}
// APIActor represents an actor that accomplishes something for an event
type APIActor struct {
ID string `json:"id,omitempty"`
Attributes map[string]string `json:"attributes,omitempty"`
}
type eventMonitoringState struct {
// `sync/atomic` expects the first word in an allocated struct to be 64-bit
// aligned on both ARM and x86-32. See https://goo.gl/zW7dgq for more details.
lastSeen int64
sync.RWMutex
sync.WaitGroup
enabled bool
C chan *APIEvents
errC chan error
listeners []chan<- *APIEvents
}
const (
maxMonitorConnRetries = 5
retryInitialWaitTime = 10.
)
var (
// ErrNoListeners is the error returned when no listeners are available
// to receive an event.
ErrNoListeners = errors.New("no listeners present to receive event")
// ErrListenerAlreadyExists is the error returned when the listerner already
// exists.
ErrListenerAlreadyExists = errors.New("listener already exists for docker events")
// ErrTLSNotSupported is the error returned when the client does not support
// TLS (this applies to the Windows named pipe client).
ErrTLSNotSupported = errors.New("tls not supported by this client")
// EOFEvent is sent when the event listener receives an EOF error.
EOFEvent = &APIEvents{
Type: "EOF",
Status: "EOF",
}
)
// AddEventListener adds a new listener to container events in the Docker API.
//
// The parameter is a channel through which events will be sent.
func (c *Client) AddEventListener(listener chan<- *APIEvents) error {
var err error
if !c.eventMonitor.isEnabled() {
err = c.eventMonitor.enableEventMonitoring(c)
if err != nil {
return err
}
}
return c.eventMonitor.addListener(listener)
}
// RemoveEventListener removes a listener from the monitor.
func (c *Client) RemoveEventListener(listener chan *APIEvents) error {
err := c.eventMonitor.removeListener(listener)
if err != nil {
return err
}
if c.eventMonitor.listernersCount() == 0 {
c.eventMonitor.disableEventMonitoring()
}
return nil
}
func (eventState *eventMonitoringState) addListener(listener chan<- *APIEvents) error {
eventState.Lock()
defer eventState.Unlock()
if listenerExists(listener, &eventState.listeners) {
return ErrListenerAlreadyExists
}
eventState.Add(1)
eventState.listeners = append(eventState.listeners, listener)
return nil
}
func (eventState *eventMonitoringState) removeListener(listener chan<- *APIEvents) error {
eventState.Lock()
defer eventState.Unlock()
if listenerExists(listener, &eventState.listeners) {
var newListeners []chan<- *APIEvents
for _, l := range eventState.listeners {
if l != listener {
newListeners = append(newListeners, l)
}
}
eventState.listeners = newListeners
eventState.Add(-1)
}
return nil
}
func (eventState *eventMonitoringState) closeListeners() {
for _, l := range eventState.listeners {
close(l)
eventState.Add(-1)
}
eventState.listeners = nil
}
func (eventState *eventMonitoringState) listernersCount() int {
eventState.RLock()
defer eventState.RUnlock()
return len(eventState.listeners)
}
func listenerExists(a chan<- *APIEvents, list *[]chan<- *APIEvents) bool {
for _, b := range *list {
if b == a {
return true
}
}
return false
}
func (eventState *eventMonitoringState) enableEventMonitoring(c *Client) error {
eventState.Lock()
defer eventState.Unlock()
if !eventState.enabled {
eventState.enabled = true
atomic.StoreInt64(&eventState.lastSeen, 0)
eventState.C = make(chan *APIEvents, 100)
eventState.errC = make(chan error, 1)
go eventState.monitorEvents(c)
}
return nil
}
func (eventState *eventMonitoringState) disableEventMonitoring() error {
eventState.Lock()
defer eventState.Unlock()
eventState.closeListeners()
eventState.Wait()
if eventState.enabled {
eventState.enabled = false
close(eventState.C)
close(eventState.errC)
}
return nil
}
func (eventState *eventMonitoringState) monitorEvents(c *Client) {
const (
noListenersTimeout = 5 * time.Second
noListenersInterval = 10 * time.Millisecond
noListenersMaxTries = noListenersTimeout / noListenersInterval
)
var err error
for i := time.Duration(0); i < noListenersMaxTries && eventState.noListeners(); i++ {
time.Sleep(10 * time.Millisecond)
}
if eventState.noListeners() {
// terminate if no listener is available after 5 seconds.
// Prevents goroutine leak when RemoveEventListener is called
// right after AddEventListener.
eventState.disableEventMonitoring()
return
}
if err = eventState.connectWithRetry(c); err != nil {
// terminate if connect failed
eventState.disableEventMonitoring()
return
}
for eventState.isEnabled() {
timeout := time.After(100 * time.Millisecond)
select {
case ev, ok := <-eventState.C:
if !ok {
return
}
if ev == EOFEvent {
eventState.disableEventMonitoring()
return
}
eventState.updateLastSeen(ev)
eventState.sendEvent(ev)
case err = <-eventState.errC:
if err == ErrNoListeners {
eventState.disableEventMonitoring()
return
} else if err != nil {
defer func() { go eventState.monitorEvents(c) }()
return
}
case <-timeout:
continue
}
}
}
func (eventState *eventMonitoringState) connectWithRetry(c *Client) error {
var retries int
eventState.RLock()
eventChan := eventState.C
errChan := eventState.errC
eventState.RUnlock()
err := c.eventHijack(atomic.LoadInt64(&eventState.lastSeen), eventChan, errChan)
for ; err != nil && retries < maxMonitorConnRetries; retries++ {
waitTime := int64(retryInitialWaitTime * math.Pow(2, float64(retries)))
time.Sleep(time.Duration(waitTime) * time.Millisecond)
eventState.RLock()
eventChan = eventState.C
errChan = eventState.errC
eventState.RUnlock()
err = c.eventHijack(atomic.LoadInt64(&eventState.lastSeen), eventChan, errChan)
}
return err
}
func (eventState *eventMonitoringState) noListeners() bool {
eventState.RLock()
defer eventState.RUnlock()
return len(eventState.listeners) == 0
}
func (eventState *eventMonitoringState) isEnabled() bool {
eventState.RLock()
defer eventState.RUnlock()
return eventState.enabled
}
func (eventState *eventMonitoringState) sendEvent(event *APIEvents) {
eventState.RLock()
defer eventState.RUnlock()
eventState.Add(1)
defer eventState.Done()
if eventState.enabled {
if len(eventState.listeners) == 0 {
eventState.errC <- ErrNoListeners
return
}
for _, listener := range eventState.listeners {
select {
case listener <- event:
default:
}
}
}
}
func (eventState *eventMonitoringState) updateLastSeen(e *APIEvents) {
eventState.Lock()
defer eventState.Unlock()
if atomic.LoadInt64(&eventState.lastSeen) < e.Time {
atomic.StoreInt64(&eventState.lastSeen, e.Time)
}
}
func (c *Client) eventHijack(startTime int64, eventChan chan *APIEvents, errChan chan error) error {
uri := "/events"
if startTime != 0 {
uri += fmt.Sprintf("?since=%d", startTime)
}
protocol := c.endpointURL.Scheme
address := c.endpointURL.Path
if protocol != "unix" && protocol != "npipe" {
protocol = "tcp"
address = c.endpointURL.Host
}
var dial net.Conn
var err error
if c.TLSConfig == nil {
dial, err = c.Dialer.Dial(protocol, address)
} else {
netDialer, ok := c.Dialer.(*net.Dialer)
if !ok {
return ErrTLSNotSupported
}
dial, err = tlsDialWithDialer(netDialer, protocol, address, c.TLSConfig)
}
if err != nil {
return err
}
conn := httputil.NewClientConn(dial, nil)
req, err := http.NewRequest("GET", uri, nil)
if err != nil {
return err
}
res, err := conn.Do(req)
if err != nil {
return err
}
go func(res *http.Response, conn *httputil.ClientConn) {
defer conn.Close()
defer res.Body.Close()
decoder := json.NewDecoder(res.Body)
for {
var event APIEvents
if err = decoder.Decode(&event); err != nil {
if err == io.EOF || err == io.ErrUnexpectedEOF {
c.eventMonitor.RLock()
if c.eventMonitor.enabled && c.eventMonitor.C == eventChan {
// Signal that we're exiting.
eventChan <- EOFEvent
}
c.eventMonitor.RUnlock()
break
}
errChan <- err
}
if event.Time == 0 {
continue
}
transformEvent(&event)
c.eventMonitor.RLock()
if c.eventMonitor.enabled && c.eventMonitor.C == eventChan {
eventChan <- &event
}
c.eventMonitor.RUnlock()
}
}(res, conn)
return nil
}
// transformEvent takes an event and determines what version it is from
// then populates both versions of the event
func transformEvent(event *APIEvents) {
// if event version is <= 1.21 there will be no Action and no Type
if event.Action == "" && event.Type == "" {
event.Action = event.Status
event.Actor.ID = event.ID
event.Actor.Attributes = map[string]string{}
switch event.Status {
case "delete", "import", "pull", "push", "tag", "untag":
event.Type = "image"
default:
event.Type = "container"
if event.From != "" {
event.Actor.Attributes["image"] = event.From
}
}
} else {
if event.Status == "" {
if event.Type == "image" || event.Type == "container" {
event.Status = event.Action
} else {
// Because just the Status has been overloaded with different Types
// if an event is not for an image or a container, we prepend the type
// to avoid problems for people relying on actions being only for
// images and containers
event.Status = event.Type + ":" + event.Action
}
}
if event.ID == "" {
event.ID = event.Actor.ID
}
if event.From == "" {
event.From = event.Actor.Attributes["image"]
}
}
}
+213
View File
@@ -0,0 +1,213 @@
// Copyright 2014 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package docker
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
)
// Exec is the type representing a `docker exec` instance and containing the
// instance ID
type Exec struct {
ID string `json:"Id,omitempty" yaml:"Id,omitempty"`
}
// CreateExecOptions specify parameters to the CreateExecContainer function.
//
// See https://goo.gl/60TeBP for more details
type CreateExecOptions struct {
AttachStdin bool `json:"AttachStdin,omitempty" yaml:"AttachStdin,omitempty" toml:"AttachStdin,omitempty"`
AttachStdout bool `json:"AttachStdout,omitempty" yaml:"AttachStdout,omitempty" toml:"AttachStdout,omitempty"`
AttachStderr bool `json:"AttachStderr,omitempty" yaml:"AttachStderr,omitempty" toml:"AttachStderr,omitempty"`
Tty bool `json:"Tty,omitempty" yaml:"Tty,omitempty" toml:"Tty,omitempty"`
Env []string `json:"Env,omitempty" yaml:"Env,omitempty" toml:"Env,omitempty"`
Cmd []string `json:"Cmd,omitempty" yaml:"Cmd,omitempty" toml:"Cmd,omitempty"`
Container string `json:"Container,omitempty" yaml:"Container,omitempty" toml:"Container,omitempty"`
User string `json:"User,omitempty" yaml:"User,omitempty" toml:"User,omitempty"`
Context context.Context `json:"-"`
Privileged bool `json:"Privileged,omitempty" yaml:"Privileged,omitempty" toml:"Privileged,omitempty"`
}
// CreateExec sets up an exec instance in a running container `id`, returning the exec
// instance, or an error in case of failure.
//
// See https://goo.gl/60TeBP for more details
func (c *Client) CreateExec(opts CreateExecOptions) (*Exec, error) {
if len(opts.Env) > 0 && c.serverAPIVersion.LessThan(apiVersion125) {
return nil, errors.New("exec configuration Env is only supported in API#1.25 and above")
}
path := fmt.Sprintf("/containers/%s/exec", opts.Container)
resp, err := c.do("POST", path, doOptions{data: opts, context: opts.Context})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return nil, &NoSuchContainer{ID: opts.Container}
}
return nil, err
}
defer resp.Body.Close()
var exec Exec
if err := json.NewDecoder(resp.Body).Decode(&exec); err != nil {
return nil, err
}
return &exec, nil
}
// StartExecOptions specify parameters to the StartExecContainer function.
//
// See https://goo.gl/1EeDWi for more details
type StartExecOptions struct {
InputStream io.Reader `qs:"-"`
OutputStream io.Writer `qs:"-"`
ErrorStream io.Writer `qs:"-"`
Detach bool `json:"Detach,omitempty" yaml:"Detach,omitempty" toml:"Detach,omitempty"`
Tty bool `json:"Tty,omitempty" yaml:"Tty,omitempty" toml:"Tty,omitempty"`
// Use raw terminal? Usually true when the container contains a TTY.
RawTerminal bool `qs:"-"`
// If set, after a successful connect, a sentinel will be sent and then the
// client will block on receive before continuing.
//
// It must be an unbuffered channel. Using a buffered channel can lead
// to unexpected behavior.
Success chan struct{} `json:"-"`
Context context.Context `json:"-"`
}
// StartExec starts a previously set up exec instance id. If opts.Detach is
// true, it returns after starting the exec command. Otherwise, it sets up an
// interactive session with the exec command.
//
// See https://goo.gl/1EeDWi for more details
func (c *Client) StartExec(id string, opts StartExecOptions) error {
cw, err := c.StartExecNonBlocking(id, opts)
if err != nil {
return err
}
if cw != nil {
return cw.Wait()
}
return nil
}
// StartExecNonBlocking starts a previously set up exec instance id. If opts.Detach is
// true, it returns after starting the exec command. Otherwise, it sets up an
// interactive session with the exec command.
//
// See https://goo.gl/1EeDWi for more details
func (c *Client) StartExecNonBlocking(id string, opts StartExecOptions) (CloseWaiter, error) {
if id == "" {
return nil, &NoSuchExec{ID: id}
}
path := fmt.Sprintf("/exec/%s/start", id)
if opts.Detach {
resp, err := c.do("POST", path, doOptions{data: opts, context: opts.Context})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return nil, &NoSuchExec{ID: id}
}
return nil, err
}
defer resp.Body.Close()
return nil, nil
}
return c.hijack("POST", path, hijackOptions{
success: opts.Success,
setRawTerminal: opts.RawTerminal,
in: opts.InputStream,
stdout: opts.OutputStream,
stderr: opts.ErrorStream,
data: opts,
})
}
// ResizeExecTTY resizes the tty session used by the exec command id. This API
// is valid only if Tty was specified as part of creating and starting the exec
// command.
//
// See https://goo.gl/Mo5bxx for more details
func (c *Client) ResizeExecTTY(id string, height, width int) error {
params := make(url.Values)
params.Set("h", strconv.Itoa(height))
params.Set("w", strconv.Itoa(width))
path := fmt.Sprintf("/exec/%s/resize?%s", id, params.Encode())
resp, err := c.do("POST", path, doOptions{})
if err != nil {
return err
}
resp.Body.Close()
return nil
}
// ExecProcessConfig is a type describing the command associated to a Exec
// instance. It's used in the ExecInspect type.
type ExecProcessConfig struct {
User string `json:"user,omitempty" yaml:"user,omitempty" toml:"user,omitempty"`
Privileged bool `json:"privileged,omitempty" yaml:"privileged,omitempty" toml:"privileged,omitempty"`
Tty bool `json:"tty,omitempty" yaml:"tty,omitempty" toml:"tty,omitempty"`
EntryPoint string `json:"entrypoint,omitempty" yaml:"entrypoint,omitempty" toml:"entrypoint,omitempty"`
Arguments []string `json:"arguments,omitempty" yaml:"arguments,omitempty" toml:"arguments,omitempty"`
}
// ExecInspect is a type with details about a exec instance, including the
// exit code if the command has finished running. It's returned by a api
// call to /exec/(id)/json
//
// See https://goo.gl/ctMUiW for more details
type ExecInspect struct {
ID string `json:"ID,omitempty" yaml:"ID,omitempty" toml:"ID,omitempty"`
ExitCode int `json:"ExitCode,omitempty" yaml:"ExitCode,omitempty" toml:"ExitCode,omitempty"`
Running bool `json:"Running,omitempty" yaml:"Running,omitempty" toml:"Running,omitempty"`
OpenStdin bool `json:"OpenStdin,omitempty" yaml:"OpenStdin,omitempty" toml:"OpenStdin,omitempty"`
OpenStderr bool `json:"OpenStderr,omitempty" yaml:"OpenStderr,omitempty" toml:"OpenStderr,omitempty"`
OpenStdout bool `json:"OpenStdout,omitempty" yaml:"OpenStdout,omitempty" toml:"OpenStdout,omitempty"`
ProcessConfig ExecProcessConfig `json:"ProcessConfig,omitempty" yaml:"ProcessConfig,omitempty" toml:"ProcessConfig,omitempty"`
ContainerID string `json:"ContainerID,omitempty" yaml:"ContainerID,omitempty" toml:"ContainerID,omitempty"`
DetachKeys string `json:"DetachKeys,omitempty" yaml:"DetachKeys,omitempty" toml:"DetachKeys,omitempty"`
CanRemove bool `json:"CanRemove,omitempty" yaml:"CanRemove,omitempty" toml:"CanRemove,omitempty"`
}
// InspectExec returns low-level information about the exec command id.
//
// See https://goo.gl/ctMUiW for more details
func (c *Client) InspectExec(id string) (*ExecInspect, error) {
path := fmt.Sprintf("/exec/%s/json", id)
resp, err := c.do("GET", path, doOptions{})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return nil, &NoSuchExec{ID: id}
}
return nil, err
}
defer resp.Body.Close()
var exec ExecInspect
if err := json.NewDecoder(resp.Body).Decode(&exec); err != nil {
return nil, err
}
return &exec, nil
}
// NoSuchExec is the error returned when a given exec instance does not exist.
type NoSuchExec struct {
ID string
}
func (err *NoSuchExec) Error() string {
return "No such exec instance: " + err.ID
}
+722
View File
@@ -0,0 +1,722 @@
// Copyright 2013 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package docker
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"time"
)
// APIImages represent an image returned in the ListImages call.
type APIImages struct {
ID string `json:"Id" yaml:"Id" toml:"Id"`
RepoTags []string `json:"RepoTags,omitempty" yaml:"RepoTags,omitempty" toml:"RepoTags,omitempty"`
Created int64 `json:"Created,omitempty" yaml:"Created,omitempty" toml:"Created,omitempty"`
Size int64 `json:"Size,omitempty" yaml:"Size,omitempty" toml:"Size,omitempty"`
VirtualSize int64 `json:"VirtualSize,omitempty" yaml:"VirtualSize,omitempty" toml:"VirtualSize,omitempty"`
ParentID string `json:"ParentId,omitempty" yaml:"ParentId,omitempty" toml:"ParentId,omitempty"`
RepoDigests []string `json:"RepoDigests,omitempty" yaml:"RepoDigests,omitempty" toml:"RepoDigests,omitempty"`
Labels map[string]string `json:"Labels,omitempty" yaml:"Labels,omitempty" toml:"Labels,omitempty"`
}
// RootFS represents the underlying layers used by an image
type RootFS struct {
Type string `json:"Type,omitempty" yaml:"Type,omitempty" toml:"Type,omitempty"`
Layers []string `json:"Layers,omitempty" yaml:"Layers,omitempty" toml:"Layers,omitempty"`
}
// Image is the type representing a docker image and its various properties
type Image struct {
ID string `json:"Id" yaml:"Id" toml:"Id"`
RepoTags []string `json:"RepoTags,omitempty" yaml:"RepoTags,omitempty" toml:"RepoTags,omitempty"`
Parent string `json:"Parent,omitempty" yaml:"Parent,omitempty" toml:"Parent,omitempty"`
Comment string `json:"Comment,omitempty" yaml:"Comment,omitempty" toml:"Comment,omitempty"`
Created time.Time `json:"Created,omitempty" yaml:"Created,omitempty" toml:"Created,omitempty"`
Container string `json:"Container,omitempty" yaml:"Container,omitempty" toml:"Container,omitempty"`
ContainerConfig Config `json:"ContainerConfig,omitempty" yaml:"ContainerConfig,omitempty" toml:"ContainerConfig,omitempty"`
DockerVersion string `json:"DockerVersion,omitempty" yaml:"DockerVersion,omitempty" toml:"DockerVersion,omitempty"`
Author string `json:"Author,omitempty" yaml:"Author,omitempty" toml:"Author,omitempty"`
Config *Config `json:"Config,omitempty" yaml:"Config,omitempty" toml:"Config,omitempty"`
Architecture string `json:"Architecture,omitempty" yaml:"Architecture,omitempty"`
Size int64 `json:"Size,omitempty" yaml:"Size,omitempty" toml:"Size,omitempty"`
VirtualSize int64 `json:"VirtualSize,omitempty" yaml:"VirtualSize,omitempty" toml:"VirtualSize,omitempty"`
RepoDigests []string `json:"RepoDigests,omitempty" yaml:"RepoDigests,omitempty" toml:"RepoDigests,omitempty"`
RootFS *RootFS `json:"RootFS,omitempty" yaml:"RootFS,omitempty" toml:"RootFS,omitempty"`
OS string `json:"Os,omitempty" yaml:"Os,omitempty" toml:"Os,omitempty"`
}
// ImagePre012 serves the same purpose as the Image type except that it is for
// earlier versions of the Docker API (pre-012 to be specific)
type ImagePre012 struct {
ID string `json:"id"`
Parent string `json:"parent,omitempty"`
Comment string `json:"comment,omitempty"`
Created time.Time `json:"created"`
Container string `json:"container,omitempty"`
ContainerConfig Config `json:"container_config,omitempty"`
DockerVersion string `json:"docker_version,omitempty"`
Author string `json:"author,omitempty"`
Config *Config `json:"config,omitempty"`
Architecture string `json:"architecture,omitempty"`
Size int64 `json:"size,omitempty"`
}
var (
// ErrNoSuchImage is the error returned when the image does not exist.
ErrNoSuchImage = errors.New("no such image")
// ErrMissingRepo is the error returned when the remote repository is
// missing.
ErrMissingRepo = errors.New("missing remote repository e.g. 'github.com/user/repo'")
// ErrMissingOutputStream is the error returned when no output stream
// is provided to some calls, like BuildImage.
ErrMissingOutputStream = errors.New("missing output stream")
// ErrMultipleContexts is the error returned when both a ContextDir and
// InputStream are provided in BuildImageOptions
ErrMultipleContexts = errors.New("image build may not be provided BOTH context dir and input stream")
// ErrMustSpecifyNames is the error rreturned when the Names field on
// ExportImagesOptions is nil or empty
ErrMustSpecifyNames = errors.New("must specify at least one name to export")
)
// ListImagesOptions specify parameters to the ListImages function.
//
// See https://goo.gl/BVzauZ for more details.
type ListImagesOptions struct {
Filters map[string][]string
All bool
Digests bool
Filter string
Context context.Context
}
// ListImages returns the list of available images in the server.
//
// See https://goo.gl/BVzauZ for more details.
func (c *Client) ListImages(opts ListImagesOptions) ([]APIImages, error) {
path := "/images/json?" + queryString(opts)
resp, err := c.do("GET", path, doOptions{context: opts.Context})
if err != nil {
return nil, err
}
defer resp.Body.Close()
var images []APIImages
if err := json.NewDecoder(resp.Body).Decode(&images); err != nil {
return nil, err
}
return images, nil
}
// ImageHistory represent a layer in an image's history returned by the
// ImageHistory call.
type ImageHistory struct {
ID string `json:"Id" yaml:"Id" toml:"Id"`
Tags []string `json:"Tags,omitempty" yaml:"Tags,omitempty" toml:"Tags,omitempty"`
Created int64 `json:"Created,omitempty" yaml:"Created,omitempty" toml:"Tags,omitempty"`
CreatedBy string `json:"CreatedBy,omitempty" yaml:"CreatedBy,omitempty" toml:"CreatedBy,omitempty"`
Size int64 `json:"Size,omitempty" yaml:"Size,omitempty" toml:"Size,omitempty"`
}
// ImageHistory returns the history of the image by its name or ID.
//
// See https://goo.gl/fYtxQa for more details.
func (c *Client) ImageHistory(name string) ([]ImageHistory, error) {
resp, err := c.do("GET", "/images/"+name+"/history", doOptions{})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return nil, ErrNoSuchImage
}
return nil, err
}
defer resp.Body.Close()
var history []ImageHistory
if err := json.NewDecoder(resp.Body).Decode(&history); err != nil {
return nil, err
}
return history, nil
}
// RemoveImage removes an image by its name or ID.
//
// See https://goo.gl/Vd2Pck for more details.
func (c *Client) RemoveImage(name string) error {
resp, err := c.do("DELETE", "/images/"+name, doOptions{})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return ErrNoSuchImage
}
return err
}
resp.Body.Close()
return nil
}
// RemoveImageOptions present the set of options available for removing an image
// from a registry.
//
// See https://goo.gl/Vd2Pck for more details.
type RemoveImageOptions struct {
Force bool `qs:"force"`
NoPrune bool `qs:"noprune"`
Context context.Context
}
// RemoveImageExtended removes an image by its name or ID.
// Extra params can be passed, see RemoveImageOptions
//
// See https://goo.gl/Vd2Pck for more details.
func (c *Client) RemoveImageExtended(name string, opts RemoveImageOptions) error {
uri := fmt.Sprintf("/images/%s?%s", name, queryString(&opts))
resp, err := c.do("DELETE", uri, doOptions{context: opts.Context})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return ErrNoSuchImage
}
return err
}
resp.Body.Close()
return nil
}
// InspectImage returns an image by its name or ID.
//
// See https://goo.gl/ncLTG8 for more details.
func (c *Client) InspectImage(name string) (*Image, error) {
resp, err := c.do("GET", "/images/"+name+"/json", doOptions{})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return nil, ErrNoSuchImage
}
return nil, err
}
defer resp.Body.Close()
var image Image
// if the caller elected to skip checking the server's version, assume it's the latest
if c.SkipServerVersionCheck || c.expectedAPIVersion.GreaterThanOrEqualTo(apiVersion112) {
if err := json.NewDecoder(resp.Body).Decode(&image); err != nil {
return nil, err
}
} else {
var imagePre012 ImagePre012
if err := json.NewDecoder(resp.Body).Decode(&imagePre012); err != nil {
return nil, err
}
image.ID = imagePre012.ID
image.Parent = imagePre012.Parent
image.Comment = imagePre012.Comment
image.Created = imagePre012.Created
image.Container = imagePre012.Container
image.ContainerConfig = imagePre012.ContainerConfig
image.DockerVersion = imagePre012.DockerVersion
image.Author = imagePre012.Author
image.Config = imagePre012.Config
image.Architecture = imagePre012.Architecture
image.Size = imagePre012.Size
}
return &image, nil
}
// PushImageOptions represents options to use in the PushImage method.
//
// See https://goo.gl/BZemGg for more details.
type PushImageOptions struct {
// Name of the image
Name string
// Tag of the image
Tag string
// Registry server to push the image
Registry string
OutputStream io.Writer `qs:"-"`
RawJSONStream bool `qs:"-"`
InactivityTimeout time.Duration `qs:"-"`
Context context.Context
}
// PushImage pushes an image to a remote registry, logging progress to w.
//
// An empty instance of AuthConfiguration may be used for unauthenticated
// pushes.
//
// See https://goo.gl/BZemGg for more details.
func (c *Client) PushImage(opts PushImageOptions, auth AuthConfiguration) error {
if opts.Name == "" {
return ErrNoSuchImage
}
headers, err := headersWithAuth(auth)
if err != nil {
return err
}
name := opts.Name
opts.Name = ""
path := "/images/" + name + "/push?" + queryString(&opts)
return c.stream("POST", path, streamOptions{
setRawTerminal: true,
rawJSONStream: opts.RawJSONStream,
headers: headers,
stdout: opts.OutputStream,
inactivityTimeout: opts.InactivityTimeout,
context: opts.Context,
})
}
// PullImageOptions present the set of options available for pulling an image
// from a registry.
//
// See https://goo.gl/qkoSsn for more details.
type PullImageOptions struct {
Repository string `qs:"fromImage"`
Tag string
// Only required for Docker Engine 1.9 or 1.10 w/ Remote API < 1.21
// and Docker Engine < 1.9
// This parameter was removed in Docker Engine 1.11
Registry string
OutputStream io.Writer `qs:"-"`
RawJSONStream bool `qs:"-"`
InactivityTimeout time.Duration `qs:"-"`
Context context.Context
}
// PullImage pulls an image from a remote registry, logging progress to
// opts.OutputStream.
//
// See https://goo.gl/qkoSsn for more details.
func (c *Client) PullImage(opts PullImageOptions, auth AuthConfiguration) error {
if opts.Repository == "" {
return ErrNoSuchImage
}
headers, err := headersWithAuth(auth)
if err != nil {
return err
}
if opts.Tag == "" && strings.Contains(opts.Repository, "@") {
parts := strings.SplitN(opts.Repository, "@", 2)
opts.Repository = parts[0]
opts.Tag = parts[1]
}
return c.createImage(queryString(&opts), headers, nil, opts.OutputStream, opts.RawJSONStream, opts.InactivityTimeout, opts.Context)
}
func (c *Client) createImage(qs string, headers map[string]string, in io.Reader, w io.Writer, rawJSONStream bool, timeout time.Duration, context context.Context) error {
path := "/images/create?" + qs
return c.stream("POST", path, streamOptions{
setRawTerminal: true,
headers: headers,
in: in,
stdout: w,
rawJSONStream: rawJSONStream,
inactivityTimeout: timeout,
context: context,
})
}
// LoadImageOptions represents the options for LoadImage Docker API Call
//
// See https://goo.gl/rEsBV3 for more details.
type LoadImageOptions struct {
InputStream io.Reader
OutputStream io.Writer
Context context.Context
}
// LoadImage imports a tarball docker image
//
// See https://goo.gl/rEsBV3 for more details.
func (c *Client) LoadImage(opts LoadImageOptions) error {
return c.stream("POST", "/images/load", streamOptions{
setRawTerminal: true,
in: opts.InputStream,
stdout: opts.OutputStream,
context: opts.Context,
})
}
// ExportImageOptions represent the options for ExportImage Docker API call.
//
// See https://goo.gl/AuySaA for more details.
type ExportImageOptions struct {
Name string
OutputStream io.Writer
InactivityTimeout time.Duration
Context context.Context
}
// ExportImage exports an image (as a tar file) into the stream.
//
// See https://goo.gl/AuySaA for more details.
func (c *Client) ExportImage(opts ExportImageOptions) error {
return c.stream("GET", fmt.Sprintf("/images/%s/get", opts.Name), streamOptions{
setRawTerminal: true,
stdout: opts.OutputStream,
inactivityTimeout: opts.InactivityTimeout,
context: opts.Context,
})
}
// ExportImagesOptions represent the options for ExportImages Docker API call
//
// See https://goo.gl/N9XlDn for more details.
type ExportImagesOptions struct {
Names []string
OutputStream io.Writer `qs:"-"`
InactivityTimeout time.Duration `qs:"-"`
Context context.Context
}
// ExportImages exports one or more images (as a tar file) into the stream
//
// See https://goo.gl/N9XlDn for more details.
func (c *Client) ExportImages(opts ExportImagesOptions) error {
if opts.Names == nil || len(opts.Names) == 0 {
return ErrMustSpecifyNames
}
return c.stream("GET", "/images/get?"+queryString(&opts), streamOptions{
setRawTerminal: true,
stdout: opts.OutputStream,
inactivityTimeout: opts.InactivityTimeout,
})
}
// ImportImageOptions present the set of informations available for importing
// an image from a source file or the stdin.
//
// See https://goo.gl/qkoSsn for more details.
type ImportImageOptions struct {
Repository string `qs:"repo"`
Source string `qs:"fromSrc"`
Tag string `qs:"tag"`
InputStream io.Reader `qs:"-"`
OutputStream io.Writer `qs:"-"`
RawJSONStream bool `qs:"-"`
InactivityTimeout time.Duration `qs:"-"`
Context context.Context
}
// ImportImage imports an image from a url, a file or stdin
//
// See https://goo.gl/qkoSsn for more details.
func (c *Client) ImportImage(opts ImportImageOptions) error {
if opts.Repository == "" {
return ErrNoSuchImage
}
if opts.Source != "-" {
opts.InputStream = nil
}
if opts.Source != "-" && !isURL(opts.Source) {
f, err := os.Open(opts.Source)
if err != nil {
return err
}
opts.InputStream = f
opts.Source = "-"
}
return c.createImage(queryString(&opts), nil, opts.InputStream, opts.OutputStream, opts.RawJSONStream, opts.InactivityTimeout, opts.Context)
}
// BuildImageOptions present the set of informations available for building an
// image from a tarfile with a Dockerfile in it.
//
// For more details about the Docker building process, see
// https://goo.gl/4nYHwV.
type BuildImageOptions struct {
Name string `qs:"t"`
Dockerfile string `qs:"dockerfile"`
NoCache bool `qs:"nocache"`
CacheFrom []string `qs:"-"`
SuppressOutput bool `qs:"q"`
Pull bool `qs:"pull"`
RmTmpContainer bool `qs:"rm"`
ForceRmTmpContainer bool `qs:"forcerm"`
RawJSONStream bool `qs:"-"`
Memory int64 `qs:"memory"`
Memswap int64 `qs:"memswap"`
CPUShares int64 `qs:"cpushares"`
CPUQuota int64 `qs:"cpuquota"`
CPUPeriod int64 `qs:"cpuperiod"`
CPUSetCPUs string `qs:"cpusetcpus"`
Labels map[string]string `qs:"labels"`
InputStream io.Reader `qs:"-"`
OutputStream io.Writer `qs:"-"`
ErrorStream io.Writer `qs:"-"`
Remote string `qs:"remote"`
Auth AuthConfiguration `qs:"-"` // for older docker X-Registry-Auth header
AuthConfigs AuthConfigurations `qs:"-"` // for newer docker X-Registry-Config header
ContextDir string `qs:"-"`
Ulimits []ULimit `qs:"-"`
BuildArgs []BuildArg `qs:"-"`
NetworkMode string `qs:"networkmode"`
InactivityTimeout time.Duration `qs:"-"`
CgroupParent string `qs:"cgroupparent"`
SecurityOpt []string `qs:"securityopt"`
Target string `gs:"target"`
Context context.Context
}
// BuildArg represents arguments that can be passed to the image when building
// it from a Dockerfile.
//
// For more details about the Docker building process, see
// https://goo.gl/4nYHwV.
type BuildArg struct {
Name string `json:"Name,omitempty" yaml:"Name,omitempty" toml:"Name,omitempty"`
Value string `json:"Value,omitempty" yaml:"Value,omitempty" toml:"Value,omitempty"`
}
// BuildImage builds an image from a tarball's url or a Dockerfile in the input
// stream.
//
// See https://goo.gl/4nYHwV for more details.
func (c *Client) BuildImage(opts BuildImageOptions) error {
if opts.OutputStream == nil {
return ErrMissingOutputStream
}
headers, err := headersWithAuth(opts.Auth, c.versionedAuthConfigs(opts.AuthConfigs))
if err != nil {
return err
}
if opts.Remote != "" && opts.Name == "" {
opts.Name = opts.Remote
}
if opts.InputStream != nil || opts.ContextDir != "" {
headers["Content-Type"] = "application/tar"
} else if opts.Remote == "" {
return ErrMissingRepo
}
if opts.ContextDir != "" {
if opts.InputStream != nil {
return ErrMultipleContexts
}
var err error
if opts.InputStream, err = createTarStream(opts.ContextDir, opts.Dockerfile); err != nil {
return err
}
}
qs := queryString(&opts)
if c.serverAPIVersion.GreaterThanOrEqualTo(apiVersion125) && len(opts.CacheFrom) > 0 {
if b, err := json.Marshal(opts.CacheFrom); err == nil {
item := url.Values(map[string][]string{})
item.Add("cachefrom", string(b))
qs = fmt.Sprintf("%s&%s", qs, item.Encode())
}
}
if len(opts.Ulimits) > 0 {
if b, err := json.Marshal(opts.Ulimits); err == nil {
item := url.Values(map[string][]string{})
item.Add("ulimits", string(b))
qs = fmt.Sprintf("%s&%s", qs, item.Encode())
}
}
if len(opts.BuildArgs) > 0 {
v := make(map[string]string)
for _, arg := range opts.BuildArgs {
v[arg.Name] = arg.Value
}
if b, err := json.Marshal(v); err == nil {
item := url.Values(map[string][]string{})
item.Add("buildargs", string(b))
qs = fmt.Sprintf("%s&%s", qs, item.Encode())
}
}
return c.stream("POST", fmt.Sprintf("/build?%s", qs), streamOptions{
setRawTerminal: true,
rawJSONStream: opts.RawJSONStream,
headers: headers,
in: opts.InputStream,
stdout: opts.OutputStream,
stderr: opts.ErrorStream,
inactivityTimeout: opts.InactivityTimeout,
context: opts.Context,
})
}
func (c *Client) versionedAuthConfigs(authConfigs AuthConfigurations) interface{} {
if c.serverAPIVersion == nil {
c.checkAPIVersion()
}
if c.serverAPIVersion != nil && c.serverAPIVersion.GreaterThanOrEqualTo(apiVersion119) {
return AuthConfigurations119(authConfigs.Configs)
}
return authConfigs
}
// TagImageOptions present the set of options to tag an image.
//
// See https://goo.gl/prHrvo for more details.
type TagImageOptions struct {
Repo string
Tag string
Force bool
Context context.Context
}
// TagImage adds a tag to the image identified by the given name.
//
// See https://goo.gl/prHrvo for more details.
func (c *Client) TagImage(name string, opts TagImageOptions) error {
if name == "" {
return ErrNoSuchImage
}
resp, err := c.do("POST", "/images/"+name+"/tag?"+queryString(&opts), doOptions{
context: opts.Context,
})
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusNotFound {
return ErrNoSuchImage
}
return err
}
func isURL(u string) bool {
p, err := url.Parse(u)
if err != nil {
return false
}
return p.Scheme == "http" || p.Scheme == "https"
}
func headersWithAuth(auths ...interface{}) (map[string]string, error) {
var headers = make(map[string]string)
for _, auth := range auths {
switch auth.(type) {
case AuthConfiguration:
var buf bytes.Buffer
if err := json.NewEncoder(&buf).Encode(auth); err != nil {
return nil, err
}
headers["X-Registry-Auth"] = base64.URLEncoding.EncodeToString(buf.Bytes())
case AuthConfigurations, AuthConfigurations119:
var buf bytes.Buffer
if err := json.NewEncoder(&buf).Encode(auth); err != nil {
return nil, err
}
headers["X-Registry-Config"] = base64.URLEncoding.EncodeToString(buf.Bytes())
}
}
return headers, nil
}
// APIImageSearch reflect the result of a search on the Docker Hub.
//
// See https://goo.gl/KLO9IZ for more details.
type APIImageSearch struct {
Description string `json:"description,omitempty" yaml:"description,omitempty" toml:"description,omitempty"`
IsOfficial bool `json:"is_official,omitempty" yaml:"is_official,omitempty" toml:"is_official,omitempty"`
IsAutomated bool `json:"is_automated,omitempty" yaml:"is_automated,omitempty" toml:"is_automated,omitempty"`
Name string `json:"name,omitempty" yaml:"name,omitempty" toml:"name,omitempty"`
StarCount int `json:"star_count,omitempty" yaml:"star_count,omitempty" toml:"star_count,omitempty"`
}
// SearchImages search the docker hub with a specific given term.
//
// See https://goo.gl/KLO9IZ for more details.
func (c *Client) SearchImages(term string) ([]APIImageSearch, error) {
resp, err := c.do("GET", "/images/search?term="+term, doOptions{})
if err != nil {
return nil, err
}
defer resp.Body.Close()
var searchResult []APIImageSearch
if err := json.NewDecoder(resp.Body).Decode(&searchResult); err != nil {
return nil, err
}
return searchResult, nil
}
// SearchImagesEx search the docker hub with a specific given term and authentication.
//
// See https://goo.gl/KLO9IZ for more details.
func (c *Client) SearchImagesEx(term string, auth AuthConfiguration) ([]APIImageSearch, error) {
headers, err := headersWithAuth(auth)
if err != nil {
return nil, err
}
resp, err := c.do("GET", "/images/search?term="+term, doOptions{
headers: headers,
})
if err != nil {
return nil, err
}
defer resp.Body.Close()
var searchResult []APIImageSearch
if err := json.NewDecoder(resp.Body).Decode(&searchResult); err != nil {
return nil, err
}
return searchResult, nil
}
// PruneImagesOptions specify parameters to the PruneImages function.
//
// See https://goo.gl/qfZlbZ for more details.
type PruneImagesOptions struct {
Filters map[string][]string
Context context.Context
}
// PruneImagesResults specify results from the PruneImages function.
//
// See https://goo.gl/qfZlbZ for more details.
type PruneImagesResults struct {
ImagesDeleted []struct{ Untagged, Deleted string }
SpaceReclaimed int64
}
// PruneImages deletes images which are unused.
//
// See https://goo.gl/qfZlbZ for more details.
func (c *Client) PruneImages(opts PruneImagesOptions) (*PruneImagesResults, error) {
path := "/images/prune?" + queryString(opts)
resp, err := c.do("POST", path, doOptions{context: opts.Context})
if err != nil {
return nil, err
}
defer resp.Body.Close()
var results PruneImagesResults
if err := json.NewDecoder(resp.Body).Decode(&results); err != nil {
return nil, err
}
return &results, nil
}
+181
View File
@@ -0,0 +1,181 @@
// Copyright 2013 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package docker
import (
"encoding/json"
"net"
"strings"
)
// Version returns version information about the docker server.
//
// See https://goo.gl/mU7yje for more details.
func (c *Client) Version() (*Env, error) {
resp, err := c.do("GET", "/version", doOptions{})
if err != nil {
return nil, err
}
defer resp.Body.Close()
var env Env
if err := env.Decode(resp.Body); err != nil {
return nil, err
}
return &env, nil
}
// DockerInfo contains information about the Docker server
//
// See https://goo.gl/bHUoz9 for more details.
type DockerInfo struct {
ID string
Containers int
ContainersRunning int
ContainersPaused int
ContainersStopped int
Images int
Driver string
DriverStatus [][2]string
SystemStatus [][2]string
Plugins PluginsInfo
MemoryLimit bool
SwapLimit bool
KernelMemory bool
CPUCfsPeriod bool `json:"CpuCfsPeriod"`
CPUCfsQuota bool `json:"CpuCfsQuota"`
CPUShares bool
CPUSet bool
IPv4Forwarding bool
BridgeNfIptables bool
BridgeNfIP6tables bool `json:"BridgeNfIp6tables"`
Debug bool
OomKillDisable bool
ExperimentalBuild bool
NFd int
NGoroutines int
SystemTime string
ExecutionDriver string
LoggingDriver string
CgroupDriver string
NEventsListener int
KernelVersion string
OperatingSystem string
OSType string
Architecture string
IndexServerAddress string
RegistryConfig *ServiceConfig
SecurityOptions []string
NCPU int
MemTotal int64
DockerRootDir string
HTTPProxy string `json:"HttpProxy"`
HTTPSProxy string `json:"HttpsProxy"`
NoProxy string
Name string
Labels []string
ServerVersion string
ClusterStore string
ClusterAdvertise string
Isolation string
InitBinary string
DefaultRuntime string
LiveRestoreEnabled bool
//Swarm swarm.Info
}
// PluginsInfo is a struct with the plugins registered with the docker daemon
//
// for more information, see: https://goo.gl/bHUoz9
type PluginsInfo struct {
// List of Volume plugins registered
Volume []string
// List of Network plugins registered
Network []string
// List of Authorization plugins registered
Authorization []string
}
// ServiceConfig stores daemon registry services configuration.
//
// for more information, see: https://goo.gl/7iFFDz
type ServiceConfig struct {
InsecureRegistryCIDRs []*NetIPNet
IndexConfigs map[string]*IndexInfo
Mirrors []string
}
// NetIPNet is the net.IPNet type, which can be marshalled and
// unmarshalled to JSON.
//
// for more information, see: https://goo.gl/7iFFDz
type NetIPNet net.IPNet
// MarshalJSON returns the JSON representation of the IPNet.
//
func (ipnet *NetIPNet) MarshalJSON() ([]byte, error) {
return json.Marshal((*net.IPNet)(ipnet).String())
}
// UnmarshalJSON sets the IPNet from a byte array of JSON.
//
func (ipnet *NetIPNet) UnmarshalJSON(b []byte) (err error) {
var ipnetStr string
if err = json.Unmarshal(b, &ipnetStr); err == nil {
var cidr *net.IPNet
if _, cidr, err = net.ParseCIDR(ipnetStr); err == nil {
*ipnet = NetIPNet(*cidr)
}
}
return
}
// IndexInfo contains information about a registry.
//
// for more information, see: https://goo.gl/7iFFDz
type IndexInfo struct {
Name string
Mirrors []string
Secure bool
Official bool
}
// Info returns system-wide information about the Docker server.
//
// See https://goo.gl/ElTHi2 for more details.
func (c *Client) Info() (*DockerInfo, error) {
resp, err := c.do("GET", "/info", doOptions{})
if err != nil {
return nil, err
}
defer resp.Body.Close()
var info DockerInfo
if err := json.NewDecoder(resp.Body).Decode(&info); err != nil {
return nil, err
}
return &info, nil
}
// ParseRepositoryTag gets the name of the repository and returns it splitted
// in two parts: the repository and the tag. It ignores the digest when it is
// present.
//
// Some examples:
//
// localhost.localdomain:5000/samalba/hipache:latest -> localhost.localdomain:5000/samalba/hipache, latest
// localhost.localdomain:5000/samalba/hipache -> localhost.localdomain:5000/samalba/hipache, ""
// busybox:latest@sha256:4a731fb46adc5cefe3ae374a8b6020fc1b6ad667a279647766e9a3cd89f6fa92 -> busybox, latest
func ParseRepositoryTag(repoTag string) (repository string, tag string) {
parts := strings.SplitN(repoTag, "@", 2)
repoTag = parts[0]
n := strings.LastIndex(repoTag, ":")
if n < 0 {
return repoTag, ""
}
if tag := repoTag[n+1:]; !strings.Contains(tag, "/") {
return repoTag[:n], tag
}
return repoTag, ""
}
+321
View File
@@ -0,0 +1,321 @@
// Copyright 2015 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package docker
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
)
// ErrNetworkAlreadyExists is the error returned by CreateNetwork when the
// network already exists.
var ErrNetworkAlreadyExists = errors.New("network already exists")
// Network represents a network.
//
// See https://goo.gl/6GugX3 for more details.
type Network struct {
Name string
ID string `json:"Id"`
Scope string
Driver string
IPAM IPAMOptions
Containers map[string]Endpoint
Options map[string]string
Internal bool
EnableIPv6 bool `json:"EnableIPv6"`
Labels map[string]string
}
// Endpoint contains network resources allocated and used for a container in a network
//
// See https://goo.gl/6GugX3 for more details.
type Endpoint struct {
Name string
ID string `json:"EndpointID"`
MacAddress string
IPv4Address string
IPv6Address string
}
// ListNetworks returns all networks.
//
// See https://goo.gl/6GugX3 for more details.
func (c *Client) ListNetworks() ([]Network, error) {
resp, err := c.do("GET", "/networks", doOptions{})
if err != nil {
return nil, err
}
defer resp.Body.Close()
var networks []Network
if err := json.NewDecoder(resp.Body).Decode(&networks); err != nil {
return nil, err
}
return networks, nil
}
// NetworkFilterOpts is an aggregation of key=value that Docker
// uses to filter networks
type NetworkFilterOpts map[string]map[string]bool
// FilteredListNetworks returns all networks with the filters applied
//
// See goo.gl/zd2mx4 for more details.
func (c *Client) FilteredListNetworks(opts NetworkFilterOpts) ([]Network, error) {
params, err := json.Marshal(opts)
if err != nil {
return nil, err
}
path := "/networks?filters=" + string(params)
resp, err := c.do("GET", path, doOptions{})
if err != nil {
return nil, err
}
defer resp.Body.Close()
var networks []Network
if err := json.NewDecoder(resp.Body).Decode(&networks); err != nil {
return nil, err
}
return networks, nil
}
// NetworkInfo returns information about a network by its ID.
//
// See https://goo.gl/6GugX3 for more details.
func (c *Client) NetworkInfo(id string) (*Network, error) {
path := "/networks/" + id
resp, err := c.do("GET", path, doOptions{})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return nil, &NoSuchNetwork{ID: id}
}
return nil, err
}
defer resp.Body.Close()
var network Network
if err := json.NewDecoder(resp.Body).Decode(&network); err != nil {
return nil, err
}
return &network, nil
}
// CreateNetworkOptions specify parameters to the CreateNetwork function and
// (for now) is the expected body of the "create network" http request message
//
// See https://goo.gl/6GugX3 for more details.
type CreateNetworkOptions struct {
Name string `json:"Name" yaml:"Name" toml:"Name"`
Driver string `json:"Driver" yaml:"Driver" toml:"Driver"`
IPAM *IPAMOptions `json:"IPAM,omitempty" yaml:"IPAM" toml:"IPAM"`
Options map[string]interface{} `json:"Options" yaml:"Options" toml:"Options"`
Labels map[string]string `json:"Labels" yaml:"Labels" toml:"Labels"`
CheckDuplicate bool `json:"CheckDuplicate" yaml:"CheckDuplicate" toml:"CheckDuplicate"`
Internal bool `json:"Internal" yaml:"Internal" toml:"Internal"`
EnableIPv6 bool `json:"EnableIPv6" yaml:"EnableIPv6" toml:"EnableIPv6"`
Context context.Context `json:"-"`
}
// IPAMOptions controls IP Address Management when creating a network
//
// See https://goo.gl/T8kRVH for more details.
type IPAMOptions struct {
Driver string `json:"Driver" yaml:"Driver" toml:"Driver"`
Config []IPAMConfig `json:"Config" yaml:"Config" toml:"Config"`
Options map[string]string `json:"Options" yaml:"Options" toml:"Options"`
}
// IPAMConfig represents IPAM configurations
//
// See https://goo.gl/T8kRVH for more details.
type IPAMConfig struct {
Subnet string `json:",omitempty"`
IPRange string `json:",omitempty"`
Gateway string `json:",omitempty"`
AuxAddress map[string]string `json:"AuxiliaryAddresses,omitempty"`
}
// CreateNetwork creates a new network, returning the network instance,
// or an error in case of failure.
//
// See https://goo.gl/6GugX3 for more details.
func (c *Client) CreateNetwork(opts CreateNetworkOptions) (*Network, error) {
resp, err := c.do(
"POST",
"/networks/create",
doOptions{
data: opts,
context: opts.Context,
},
)
if err != nil {
return nil, err
}
defer resp.Body.Close()
type createNetworkResponse struct {
ID string
}
var (
network Network
cnr createNetworkResponse
)
if err := json.NewDecoder(resp.Body).Decode(&cnr); err != nil {
return nil, err
}
network.Name = opts.Name
network.ID = cnr.ID
network.Driver = opts.Driver
return &network, nil
}
// RemoveNetwork removes a network or returns an error in case of failure.
//
// See https://goo.gl/6GugX3 for more details.
func (c *Client) RemoveNetwork(id string) error {
resp, err := c.do("DELETE", "/networks/"+id, doOptions{})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return &NoSuchNetwork{ID: id}
}
return err
}
resp.Body.Close()
return nil
}
// NetworkConnectionOptions specify parameters to the ConnectNetwork and
// DisconnectNetwork function.
//
// See https://goo.gl/RV7BJU for more details.
type NetworkConnectionOptions struct {
Container string
// EndpointConfig is only applicable to the ConnectNetwork call
EndpointConfig *EndpointConfig `json:"EndpointConfig,omitempty"`
// Force is only applicable to the DisconnectNetwork call
Force bool
Context context.Context `json:"-"`
}
// EndpointConfig stores network endpoint details
//
// See https://goo.gl/RV7BJU for more details.
type EndpointConfig struct {
IPAMConfig *EndpointIPAMConfig `json:"IPAMConfig,omitempty" yaml:"IPAMConfig,omitempty" toml:"IPAMConfig,omitempty"`
Links []string `json:"Links,omitempty" yaml:"Links,omitempty" toml:"Links,omitempty"`
Aliases []string `json:"Aliases,omitempty" yaml:"Aliases,omitempty" toml:"Aliases,omitempty"`
NetworkID string `json:"NetworkID,omitempty" yaml:"NetworkID,omitempty" toml:"NetworkID,omitempty"`
EndpointID string `json:"EndpointID,omitempty" yaml:"EndpointID,omitempty" toml:"EndpointID,omitempty"`
Gateway string `json:"Gateway,omitempty" yaml:"Gateway,omitempty" toml:"Gateway,omitempty"`
IPAddress string `json:"IPAddress,omitempty" yaml:"IPAddress,omitempty" toml:"IPAddress,omitempty"`
IPPrefixLen int `json:"IPPrefixLen,omitempty" yaml:"IPPrefixLen,omitempty" toml:"IPPrefixLen,omitempty"`
IPv6Gateway string `json:"IPv6Gateway,omitempty" yaml:"IPv6Gateway,omitempty" toml:"IPv6Gateway,omitempty"`
GlobalIPv6Address string `json:"GlobalIPv6Address,omitempty" yaml:"GlobalIPv6Address,omitempty" toml:"GlobalIPv6Address,omitempty"`
GlobalIPv6PrefixLen int `json:"GlobalIPv6PrefixLen,omitempty" yaml:"GlobalIPv6PrefixLen,omitempty" toml:"GlobalIPv6PrefixLen,omitempty"`
MacAddress string `json:"MacAddress,omitempty" yaml:"MacAddress,omitempty" toml:"MacAddress,omitempty"`
}
// EndpointIPAMConfig represents IPAM configurations for an
// endpoint
//
// See https://goo.gl/RV7BJU for more details.
type EndpointIPAMConfig struct {
IPv4Address string `json:",omitempty"`
IPv6Address string `json:",omitempty"`
}
// ConnectNetwork adds a container to a network or returns an error in case of
// failure.
//
// See https://goo.gl/6GugX3 for more details.
func (c *Client) ConnectNetwork(id string, opts NetworkConnectionOptions) error {
resp, err := c.do("POST", "/networks/"+id+"/connect", doOptions{
data: opts,
context: opts.Context,
})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return &NoSuchNetworkOrContainer{NetworkID: id, ContainerID: opts.Container}
}
return err
}
resp.Body.Close()
return nil
}
// DisconnectNetwork removes a container from a network or returns an error in
// case of failure.
//
// See https://goo.gl/6GugX3 for more details.
func (c *Client) DisconnectNetwork(id string, opts NetworkConnectionOptions) error {
resp, err := c.do("POST", "/networks/"+id+"/disconnect", doOptions{data: opts})
if err != nil {
if e, ok := err.(*Error); ok && e.Status == http.StatusNotFound {
return &NoSuchNetworkOrContainer{NetworkID: id, ContainerID: opts.Container}
}
return err
}
resp.Body.Close()
return nil
}
// PruneNetworksOptions specify parameters to the PruneNetworks function.
//
// See https://goo.gl/kX0S9h for more details.
type PruneNetworksOptions struct {
Filters map[string][]string
Context context.Context
}
// PruneNetworksResults specify results from the PruneNetworks function.
//
// See https://goo.gl/kX0S9h for more details.
type PruneNetworksResults struct {
NetworksDeleted []string
}
// PruneNetworks deletes networks which are unused.
//
// See https://goo.gl/kX0S9h for more details.
func (c *Client) PruneNetworks(opts PruneNetworksOptions) (*PruneNetworksResults, error) {
path := "/networks/prune?" + queryString(opts)
resp, err := c.do("POST", path, doOptions{context: opts.Context})
if err != nil {
return nil, err
}
defer resp.Body.Close()
var results PruneNetworksResults
if err := json.NewDecoder(resp.Body).Decode(&results); err != nil {
return nil, err
}
return &results, nil
}
// NoSuchNetwork is the error returned when a given network does not exist.
type NoSuchNetwork struct {
ID string
}
func (err *NoSuchNetwork) Error() string {
return fmt.Sprintf("No such network: %s", err.ID)
}
// NoSuchNetworkOrContainer is the error returned when a given network or
// container does not exist.
type NoSuchNetworkOrContainer struct {
NetworkID string
ContainerID string
}
func (err *NoSuchNetworkOrContainer) Error() string {
return fmt.Sprintf("No such network (%s) or container (%s)", err.NetworkID, err.ContainerID)
}
+48
View File
@@ -0,0 +1,48 @@
package opts
import (
"fmt"
"os"
"runtime"
"strings"
"github.com/pkg/errors"
)
// ValidateEnv validates an environment variable and returns it.
// If no value is specified, it returns the current value using os.Getenv.
//
// As on ParseEnvFile and related to #16585, environment variable names
// are not validate what so ever, it's up to application inside docker
// to validate them or not.
//
// The only validation here is to check if name is empty, per #25099
func ValidateEnv(val string) (string, error) {
arr := strings.Split(val, "=")
if arr[0] == "" {
return "", errors.Errorf("invalid environment variable: %s", val)
}
if len(arr) > 1 {
return val, nil
}
if !doesEnvExist(val) {
return val, nil
}
return fmt.Sprintf("%s=%s", val, os.Getenv(val)), nil
}
func doesEnvExist(name string) bool {
for _, entry := range os.Environ() {
parts := strings.SplitN(entry, "=", 2)
if runtime.GOOS == "windows" {
// Environment variable are case-insensitive on Windows. PaTh, path and PATH are equivalent.
if strings.EqualFold(parts[0], name) {
return true
}
}
if parts[0] == name {
return true
}
}
return false
}
+165
View File
@@ -0,0 +1,165 @@
package opts
import (
"fmt"
"net"
"net/url"
"strconv"
"strings"
)
var (
// DefaultHTTPPort Default HTTP Port used if only the protocol is provided to -H flag e.g. dockerd -H tcp://
// These are the IANA registered port numbers for use with Docker
// see http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=docker
DefaultHTTPPort = 2375 // Default HTTP Port
// DefaultTLSHTTPPort Default HTTP Port used when TLS enabled
DefaultTLSHTTPPort = 2376 // Default TLS encrypted HTTP Port
// DefaultUnixSocket Path for the unix socket.
// Docker daemon by default always listens on the default unix socket
DefaultUnixSocket = "/var/run/docker.sock"
// DefaultTCPHost constant defines the default host string used by docker on Windows
DefaultTCPHost = fmt.Sprintf("tcp://%s:%d", DefaultHTTPHost, DefaultHTTPPort)
// DefaultTLSHost constant defines the default host string used by docker for TLS sockets
DefaultTLSHost = fmt.Sprintf("tcp://%s:%d", DefaultHTTPHost, DefaultTLSHTTPPort)
// DefaultNamedPipe defines the default named pipe used by docker on Windows
DefaultNamedPipe = `//./pipe/docker_engine`
)
// ValidateHost validates that the specified string is a valid host and returns it.
func ValidateHost(val string) (string, error) {
host := strings.TrimSpace(val)
// The empty string means default and is not handled by parseDaemonHost
if host != "" {
_, err := parseDaemonHost(host)
if err != nil {
return val, err
}
}
// Note: unlike most flag validators, we don't return the mutated value here
// we need to know what the user entered later (using ParseHost) to adjust for TLS
return val, nil
}
// ParseHost and set defaults for a Daemon host string
func ParseHost(defaultToTLS bool, val string) (string, error) {
host := strings.TrimSpace(val)
if host == "" {
if defaultToTLS {
host = DefaultTLSHost
} else {
host = DefaultHost
}
} else {
var err error
host, err = parseDaemonHost(host)
if err != nil {
return val, err
}
}
return host, nil
}
// parseDaemonHost parses the specified address and returns an address that will be used as the host.
// Depending of the address specified, this may return one of the global Default* strings defined in hosts.go.
func parseDaemonHost(addr string) (string, error) {
addrParts := strings.SplitN(addr, "://", 2)
if len(addrParts) == 1 && addrParts[0] != "" {
addrParts = []string{"tcp", addrParts[0]}
}
switch addrParts[0] {
case "tcp":
return ParseTCPAddr(addrParts[1], DefaultTCPHost)
case "unix":
return parseSimpleProtoAddr("unix", addrParts[1], DefaultUnixSocket)
case "npipe":
return parseSimpleProtoAddr("npipe", addrParts[1], DefaultNamedPipe)
case "fd":
return addr, nil
default:
return "", fmt.Errorf("Invalid bind address format: %s", addr)
}
}
// parseSimpleProtoAddr parses and validates that the specified address is a valid
// socket address for simple protocols like unix and npipe. It returns a formatted
// socket address, either using the address parsed from addr, or the contents of
// defaultAddr if addr is a blank string.
func parseSimpleProtoAddr(proto, addr, defaultAddr string) (string, error) {
addr = strings.TrimPrefix(addr, proto+"://")
if strings.Contains(addr, "://") {
return "", fmt.Errorf("Invalid proto, expected %s: %s", proto, addr)
}
if addr == "" {
addr = defaultAddr
}
return fmt.Sprintf("%s://%s", proto, addr), nil
}
// ParseTCPAddr parses and validates that the specified address is a valid TCP
// address. It returns a formatted TCP address, either using the address parsed
// from tryAddr, or the contents of defaultAddr if tryAddr is a blank string.
// tryAddr is expected to have already been Trim()'d
// defaultAddr must be in the full `tcp://host:port` form
func ParseTCPAddr(tryAddr string, defaultAddr string) (string, error) {
if tryAddr == "" || tryAddr == "tcp://" {
return defaultAddr, nil
}
addr := strings.TrimPrefix(tryAddr, "tcp://")
if strings.Contains(addr, "://") || addr == "" {
return "", fmt.Errorf("Invalid proto, expected tcp: %s", tryAddr)
}
defaultAddr = strings.TrimPrefix(defaultAddr, "tcp://")
defaultHost, defaultPort, err := net.SplitHostPort(defaultAddr)
if err != nil {
return "", err
}
// url.Parse fails for trailing colon on IPv6 brackets on Go 1.5, but
// not 1.4. See https://github.com/golang/go/issues/12200 and
// https://github.com/golang/go/issues/6530.
if strings.HasSuffix(addr, "]:") {
addr += defaultPort
}
u, err := url.Parse("tcp://" + addr)
if err != nil {
return "", err
}
host, port, err := net.SplitHostPort(u.Host)
if err != nil {
// try port addition once
host, port, err = net.SplitHostPort(net.JoinHostPort(u.Host, defaultPort))
}
if err != nil {
return "", fmt.Errorf("Invalid bind address format: %s", tryAddr)
}
if host == "" {
host = defaultHost
}
if port == "" {
port = defaultPort
}
p, err := strconv.Atoi(port)
if err != nil && p == 0 {
return "", fmt.Errorf("Invalid bind address format: %s", tryAddr)
}
return fmt.Sprintf("tcp://%s%s", net.JoinHostPort(host, port), u.Path), nil
}
// ValidateExtraHost validates that the specified string is a valid extrahost and returns it.
// ExtraHost is in the form of name:ip where the ip has to be a valid ip (IPv4 or IPv6).
func ValidateExtraHost(val string) (string, error) {
// allow for IPv6 addresses in extra hosts by only splitting on first ":"
arr := strings.SplitN(val, ":", 2)
if len(arr) != 2 || len(arr[0]) == 0 {
return "", fmt.Errorf("bad format for add-host: %q", val)
}
if _, err := ValidateIPAddress(arr[1]); err != nil {
return "", fmt.Errorf("invalid IP address in add-host: %q", arr[1])
}
return val, nil
}
+8
View File
@@ -0,0 +1,8 @@
// +build !windows
package opts
import "fmt"
// DefaultHost constant defines the default host string used by docker on other hosts than Windows
var DefaultHost = fmt.Sprintf("unix://%s", DefaultUnixSocket)
+4
View File
@@ -0,0 +1,4 @@
package opts
// DefaultHost constant defines the default host string used by docker on Windows
var DefaultHost = "npipe://" + DefaultNamedPipe
+47
View File
@@ -0,0 +1,47 @@
package opts
import (
"fmt"
"net"
)
// IPOpt holds an IP. It is used to store values from CLI flags.
type IPOpt struct {
*net.IP
}
// NewIPOpt creates a new IPOpt from a reference net.IP and a
// string representation of an IP. If the string is not a valid
// IP it will fallback to the specified reference.
func NewIPOpt(ref *net.IP, defaultVal string) *IPOpt {
o := &IPOpt{
IP: ref,
}
o.Set(defaultVal)
return o
}
// Set sets an IPv4 or IPv6 address from a given string. If the given
// string is not parsable as an IP address it returns an error.
func (o *IPOpt) Set(val string) error {
ip := net.ParseIP(val)
if ip == nil {
return fmt.Errorf("%s is not an ip address", val)
}
*o.IP = ip
return nil
}
// String returns the IP address stored in the IPOpt. If stored IP is a
// nil pointer, it returns an empty string.
func (o *IPOpt) String() string {
if *o.IP == nil {
return ""
}
return o.IP.String()
}
// Type returns the type of the option
func (o *IPOpt) Type() string {
return "ip"
}
+337
View File
@@ -0,0 +1,337 @@
package opts
import (
"fmt"
"net"
"path"
"regexp"
"strings"
units "github.com/docker/go-units"
)
var (
alphaRegexp = regexp.MustCompile(`[a-zA-Z]`)
domainRegexp = regexp.MustCompile(`^(:?(:?[a-zA-Z0-9]|(:?[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9]))(:?\.(:?[a-zA-Z0-9]|(:?[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])))*)\.?\s*$`)
)
// ListOpts holds a list of values and a validation function.
type ListOpts struct {
values *[]string
validator ValidatorFctType
}
// NewListOpts creates a new ListOpts with the specified validator.
func NewListOpts(validator ValidatorFctType) ListOpts {
var values []string
return *NewListOptsRef(&values, validator)
}
// NewListOptsRef creates a new ListOpts with the specified values and validator.
func NewListOptsRef(values *[]string, validator ValidatorFctType) *ListOpts {
return &ListOpts{
values: values,
validator: validator,
}
}
func (opts *ListOpts) String() string {
if len(*opts.values) == 0 {
return ""
}
return fmt.Sprintf("%v", *opts.values)
}
// Set validates if needed the input value and adds it to the
// internal slice.
func (opts *ListOpts) Set(value string) error {
if opts.validator != nil {
v, err := opts.validator(value)
if err != nil {
return err
}
value = v
}
(*opts.values) = append((*opts.values), value)
return nil
}
// Delete removes the specified element from the slice.
func (opts *ListOpts) Delete(key string) {
for i, k := range *opts.values {
if k == key {
(*opts.values) = append((*opts.values)[:i], (*opts.values)[i+1:]...)
return
}
}
}
// GetMap returns the content of values in a map in order to avoid
// duplicates.
func (opts *ListOpts) GetMap() map[string]struct{} {
ret := make(map[string]struct{})
for _, k := range *opts.values {
ret[k] = struct{}{}
}
return ret
}
// GetAll returns the values of slice.
func (opts *ListOpts) GetAll() []string {
return (*opts.values)
}
// GetAllOrEmpty returns the values of the slice
// or an empty slice when there are no values.
func (opts *ListOpts) GetAllOrEmpty() []string {
v := *opts.values
if v == nil {
return make([]string, 0)
}
return v
}
// Get checks the existence of the specified key.
func (opts *ListOpts) Get(key string) bool {
for _, k := range *opts.values {
if k == key {
return true
}
}
return false
}
// Len returns the amount of element in the slice.
func (opts *ListOpts) Len() int {
return len((*opts.values))
}
// Type returns a string name for this Option type
func (opts *ListOpts) Type() string {
return "list"
}
// WithValidator returns the ListOpts with validator set.
func (opts *ListOpts) WithValidator(validator ValidatorFctType) *ListOpts {
opts.validator = validator
return opts
}
// NamedOption is an interface that list and map options
// with names implement.
type NamedOption interface {
Name() string
}
// NamedListOpts is a ListOpts with a configuration name.
// This struct is useful to keep reference to the assigned
// field name in the internal configuration struct.
type NamedListOpts struct {
name string
ListOpts
}
var _ NamedOption = &NamedListOpts{}
// NewNamedListOptsRef creates a reference to a new NamedListOpts struct.
func NewNamedListOptsRef(name string, values *[]string, validator ValidatorFctType) *NamedListOpts {
return &NamedListOpts{
name: name,
ListOpts: *NewListOptsRef(values, validator),
}
}
// Name returns the name of the NamedListOpts in the configuration.
func (o *NamedListOpts) Name() string {
return o.name
}
// MapOpts holds a map of values and a validation function.
type MapOpts struct {
values map[string]string
validator ValidatorFctType
}
// Set validates if needed the input value and add it to the
// internal map, by splitting on '='.
func (opts *MapOpts) Set(value string) error {
if opts.validator != nil {
v, err := opts.validator(value)
if err != nil {
return err
}
value = v
}
vals := strings.SplitN(value, "=", 2)
if len(vals) == 1 {
(opts.values)[vals[0]] = ""
} else {
(opts.values)[vals[0]] = vals[1]
}
return nil
}
// GetAll returns the values of MapOpts as a map.
func (opts *MapOpts) GetAll() map[string]string {
return opts.values
}
func (opts *MapOpts) String() string {
return fmt.Sprintf("%v", opts.values)
}
// Type returns a string name for this Option type
func (opts *MapOpts) Type() string {
return "map"
}
// NewMapOpts creates a new MapOpts with the specified map of values and a validator.
func NewMapOpts(values map[string]string, validator ValidatorFctType) *MapOpts {
if values == nil {
values = make(map[string]string)
}
return &MapOpts{
values: values,
validator: validator,
}
}
// NamedMapOpts is a MapOpts struct with a configuration name.
// This struct is useful to keep reference to the assigned
// field name in the internal configuration struct.
type NamedMapOpts struct {
name string
MapOpts
}
var _ NamedOption = &NamedMapOpts{}
// NewNamedMapOpts creates a reference to a new NamedMapOpts struct.
func NewNamedMapOpts(name string, values map[string]string, validator ValidatorFctType) *NamedMapOpts {
return &NamedMapOpts{
name: name,
MapOpts: *NewMapOpts(values, validator),
}
}
// Name returns the name of the NamedMapOpts in the configuration.
func (o *NamedMapOpts) Name() string {
return o.name
}
// ValidatorFctType defines a validator function that returns a validated string and/or an error.
type ValidatorFctType func(val string) (string, error)
// ValidatorFctListType defines a validator function that returns a validated list of string and/or an error
type ValidatorFctListType func(val string) ([]string, error)
// ValidateIPAddress validates an Ip address.
func ValidateIPAddress(val string) (string, error) {
var ip = net.ParseIP(strings.TrimSpace(val))
if ip != nil {
return ip.String(), nil
}
return "", fmt.Errorf("%s is not an ip address", val)
}
// ValidateDNSSearch validates domain for resolvconf search configuration.
// A zero length domain is represented by a dot (.).
func ValidateDNSSearch(val string) (string, error) {
if val = strings.Trim(val, " "); val == "." {
return val, nil
}
return validateDomain(val)
}
func validateDomain(val string) (string, error) {
if alphaRegexp.FindString(val) == "" {
return "", fmt.Errorf("%s is not a valid domain", val)
}
ns := domainRegexp.FindSubmatch([]byte(val))
if len(ns) > 0 && len(ns[1]) < 255 {
return string(ns[1]), nil
}
return "", fmt.Errorf("%s is not a valid domain", val)
}
// ValidateLabel validates that the specified string is a valid label, and returns it.
// Labels are in the form on key=value.
func ValidateLabel(val string) (string, error) {
if strings.Count(val, "=") < 1 {
return "", fmt.Errorf("bad attribute format: %s", val)
}
return val, nil
}
// ValidateSingleGenericResource validates that a single entry in the
// generic resource list is valid.
// i.e 'GPU=UID1' is valid however 'GPU:UID1' or 'UID1' isn't
func ValidateSingleGenericResource(val string) (string, error) {
if strings.Count(val, "=") < 1 {
return "", fmt.Errorf("invalid node-generic-resource format `%s` expected `name=value`", val)
}
return val, nil
}
// ParseLink parses and validates the specified string as a link format (name:alias)
func ParseLink(val string) (string, string, error) {
if val == "" {
return "", "", fmt.Errorf("empty string specified for links")
}
arr := strings.Split(val, ":")
if len(arr) > 2 {
return "", "", fmt.Errorf("bad format for links: %s", val)
}
if len(arr) == 1 {
return val, val, nil
}
// This is kept because we can actually get a HostConfig with links
// from an already created container and the format is not `foo:bar`
// but `/foo:/c1/bar`
if strings.HasPrefix(arr[0], "/") {
_, alias := path.Split(arr[1])
return arr[0][1:], alias, nil
}
return arr[0], arr[1], nil
}
// MemBytes is a type for human readable memory bytes (like 128M, 2g, etc)
type MemBytes int64
// String returns the string format of the human readable memory bytes
func (m *MemBytes) String() string {
// NOTE: In spf13/pflag/flag.go, "0" is considered as "zero value" while "0 B" is not.
// We return "0" in case value is 0 here so that the default value is hidden.
// (Sometimes "default 0 B" is actually misleading)
if m.Value() != 0 {
return units.BytesSize(float64(m.Value()))
}
return "0"
}
// Set sets the value of the MemBytes by passing a string
func (m *MemBytes) Set(value string) error {
val, err := units.RAMInBytes(value)
*m = MemBytes(val)
return err
}
// Type returns the type
func (m *MemBytes) Type() string {
return "bytes"
}
// Value returns the value in int64
func (m *MemBytes) Value() int64 {
return int64(*m)
}
// UnmarshalJSON is the customized unmarshaler for MemBytes
func (m *MemBytes) UnmarshalJSON(s []byte) error {
if len(s) <= 2 || s[0] != '"' || s[len(s)-1] != '"' {
return fmt.Errorf("invalid size: %q", s)
}
val, err := units.RAMInBytes(string(s[1 : len(s)-1]))
*m = MemBytes(val)
return err
}
+22
View File
@@ -0,0 +1,22 @@
// +build !windows
package opts
import (
"fmt"
"strings"
"github.com/pkg/errors"
)
// DefaultHTTPHost Default HTTP Host used if only port is provided to -H flag e.g. dockerd -H tcp://:8080
const DefaultHTTPHost = "localhost"
// MountParser parses mount path.
func MountParser(mount string) (source, destination string, err error) {
sd := strings.Split(mount, ":")
if len(sd) == 2 {
return sd[0], sd[1], nil
}
return "", "", errors.Wrap(fmt.Errorf("invalid mount format: got %s, expected <src>:<dst>", mount), "")
}
+72
View File
@@ -0,0 +1,72 @@
package opts
import (
"fmt"
"strings"
"github.com/pkg/errors"
)
// TODO Windows. Identify bug in GOLang 1.5.1+ and/or Windows Server 2016 TP5.
// @jhowardmsft, @swernli.
//
// On Windows, this mitigates a problem with the default options of running
// a docker client against a local docker daemon on TP5.
//
// What was found that if the default host is "localhost", even if the client
// (and daemon as this is local) is not physically on a network, and the DNS
// cache is flushed (ipconfig /flushdns), then the client will pause for
// exactly one second when connecting to the daemon for calls. For example
// using docker run windowsservercore cmd, the CLI will send a create followed
// by an attach. You see the delay between the attach finishing and the attach
// being seen by the daemon.
//
// Here's some daemon debug logs with additional debug spew put in. The
// AfterWriteJSON log is the very last thing the daemon does as part of the
// create call. The POST /attach is the second CLI call. Notice the second
// time gap.
//
// time="2015-11-06T13:38:37.259627400-08:00" level=debug msg="After createRootfs"
// time="2015-11-06T13:38:37.263626300-08:00" level=debug msg="After setHostConfig"
// time="2015-11-06T13:38:37.267631200-08:00" level=debug msg="before createContainerPl...."
// time="2015-11-06T13:38:37.271629500-08:00" level=debug msg=ToDiskLocking....
// time="2015-11-06T13:38:37.275643200-08:00" level=debug msg="loggin event...."
// time="2015-11-06T13:38:37.277627600-08:00" level=debug msg="logged event...."
// time="2015-11-06T13:38:37.279631800-08:00" level=debug msg="In defer func"
// time="2015-11-06T13:38:37.282628100-08:00" level=debug msg="After daemon.create"
// time="2015-11-06T13:38:37.286651700-08:00" level=debug msg="return 2"
// time="2015-11-06T13:38:37.289629500-08:00" level=debug msg="Returned from daemon.ContainerCreate"
// time="2015-11-06T13:38:37.311629100-08:00" level=debug msg="After WriteJSON"
// ... 1 second gap here....
// time="2015-11-06T13:38:38.317866200-08:00" level=debug msg="Calling POST /v1.22/containers/984758282b842f779e805664b2c95d563adc9a979c8a3973e68c807843ee4757/attach"
// time="2015-11-06T13:38:38.326882500-08:00" level=info msg="POST /v1.22/containers/984758282b842f779e805664b2c95d563adc9a979c8a3973e68c807843ee4757/attach?stderr=1&stdin=1&stdout=1&stream=1"
//
// We suspect this is either a bug introduced in GOLang 1.5.1, or that a change
// in GOLang 1.5.1 (from 1.4.3) is exposing a bug in Windows. In theory,
// the Windows networking stack is supposed to resolve "localhost" internally,
// without hitting DNS, or even reading the hosts file (which is why localhost
// is commented out in the hosts file on Windows).
//
// We have validated that working around this using the actual IPv4 localhost
// address does not cause the delay.
//
// This does not occur with the docker client built with 1.4.3 on the same
// Windows build, regardless of whether the daemon is built using 1.5.1
// or 1.4.3. It does not occur on Linux. We also verified we see the same thing
// on a cross-compiled Windows binary (from Linux).
//
// Final note: This is a mitigation, not a 'real' fix. It is still susceptible
// to the delay if a user were to do 'docker run -H=tcp://localhost:2375...'
// explicitly.
// DefaultHTTPHost Default HTTP Host used if only port is provided to -H flag e.g. dockerd -H tcp://:8080
const DefaultHTTPHost = "127.0.0.1"
// MountParser parses mount path.
func MountParser(mount string) (source, destination string, err error) {
sd := strings.Split(mount, ":")
if len(sd) == 3 {
return sd[0] + sd[1], sd[2], nil
}
return "", "", errors.Wrap(fmt.Errorf("invalid mount format: got %s, expected <drive>:<src>:<dst>", mount), "")
}
+37
View File
@@ -0,0 +1,37 @@
package opts
// QuotedString is a string that may have extra quotes around the value. The
// quotes are stripped from the value.
type QuotedString struct {
value *string
}
// Set sets a new value
func (s *QuotedString) Set(val string) error {
*s.value = trimQuotes(val)
return nil
}
// Type returns the type of the value
func (s *QuotedString) Type() string {
return "string"
}
func (s *QuotedString) String() string {
return *s.value
}
func trimQuotes(value string) string {
lastIndex := len(value) - 1
for _, char := range []byte{'\'', '"'} {
if value[0] == char && value[lastIndex] == char {
return value[1:lastIndex]
}
}
return value
}
// NewQuotedString returns a new quoted string option
func NewQuotedString(value *string) *QuotedString {
return &QuotedString{value: value}
}
+79
View File
@@ -0,0 +1,79 @@
package opts
import (
"fmt"
"strings"
"github.com/ory/dockertest/v3/docker/types"
)
// RuntimeOpt defines a map of Runtimes
type RuntimeOpt struct {
name string
stockRuntimeName string
values *map[string]types.Runtime
}
// NewNamedRuntimeOpt creates a new RuntimeOpt
func NewNamedRuntimeOpt(name string, ref *map[string]types.Runtime, stockRuntime string) *RuntimeOpt {
if ref == nil {
ref = &map[string]types.Runtime{}
}
return &RuntimeOpt{name: name, values: ref, stockRuntimeName: stockRuntime}
}
// Name returns the name of the NamedListOpts in the configuration.
func (o *RuntimeOpt) Name() string {
return o.name
}
// Set validates and updates the list of Runtimes
func (o *RuntimeOpt) Set(val string) error {
parts := strings.SplitN(val, "=", 2)
if len(parts) != 2 {
return fmt.Errorf("invalid runtime argument: %s", val)
}
parts[0] = strings.TrimSpace(parts[0])
parts[1] = strings.TrimSpace(parts[1])
if parts[0] == "" || parts[1] == "" {
return fmt.Errorf("invalid runtime argument: %s", val)
}
parts[0] = strings.ToLower(parts[0])
if parts[0] == o.stockRuntimeName {
return fmt.Errorf("runtime name '%s' is reserved", o.stockRuntimeName)
}
if _, ok := (*o.values)[parts[0]]; ok {
return fmt.Errorf("runtime '%s' was already defined", parts[0])
}
(*o.values)[parts[0]] = types.Runtime{Path: parts[1]}
return nil
}
// String returns Runtime values as a string.
func (o *RuntimeOpt) String() string {
var out []string
for k := range *o.values {
out = append(out, k)
}
return fmt.Sprintf("%v", out)
}
// GetMap returns a map of Runtimes (name: path)
func (o *RuntimeOpt) GetMap() map[string]types.Runtime {
if o.values != nil {
return *o.values
}
return map[string]types.Runtime{}
}
// Type returns the type of the option
func (o *RuntimeOpt) Type() string {
return "runtime"
}
+81
View File
@@ -0,0 +1,81 @@
package opts
import (
"fmt"
"github.com/docker/go-units"
)
// UlimitOpt defines a map of Ulimits
type UlimitOpt struct {
values *map[string]*units.Ulimit
}
// NewUlimitOpt creates a new UlimitOpt
func NewUlimitOpt(ref *map[string]*units.Ulimit) *UlimitOpt {
if ref == nil {
ref = &map[string]*units.Ulimit{}
}
return &UlimitOpt{ref}
}
// Set validates a Ulimit and sets its name as a key in UlimitOpt
func (o *UlimitOpt) Set(val string) error {
l, err := units.ParseUlimit(val)
if err != nil {
return err
}
(*o.values)[l.Name] = l
return nil
}
// String returns Ulimit values as a string.
func (o *UlimitOpt) String() string {
var out []string
for _, v := range *o.values {
out = append(out, v.String())
}
return fmt.Sprintf("%v", out)
}
// GetList returns a slice of pointers to Ulimits.
func (o *UlimitOpt) GetList() []*units.Ulimit {
var ulimits []*units.Ulimit
for _, v := range *o.values {
ulimits = append(ulimits, v)
}
return ulimits
}
// Type returns the option type
func (o *UlimitOpt) Type() string {
return "ulimit"
}
// NamedUlimitOpt defines a named map of Ulimits
type NamedUlimitOpt struct {
name string
UlimitOpt
}
var _ NamedOption = &NamedUlimitOpt{}
// NewNamedUlimitOpt creates a new NamedUlimitOpt
func NewNamedUlimitOpt(name string, ref *map[string]*units.Ulimit) *NamedUlimitOpt {
if ref == nil {
ref = &map[string]*units.Ulimit{}
}
return &NamedUlimitOpt{
name: name,
UlimitOpt: *NewUlimitOpt(ref),
}
}
// Name returns the option name
func (o *NamedUlimitOpt) Name() string {
return o.name
}
+1
View File
@@ -0,0 +1 @@
This code provides helper functions for dealing with archive files.
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,92 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"archive/tar"
"os"
"path/filepath"
"strings"
"github.com/ory/dockertest/v3/docker/pkg/system"
"golang.org/x/sys/unix"
)
func getWhiteoutConverter(format WhiteoutFormat) tarWhiteoutConverter {
if format == OverlayWhiteoutFormat {
return overlayWhiteoutConverter{}
}
return nil
}
type overlayWhiteoutConverter struct{}
func (overlayWhiteoutConverter) ConvertWrite(hdr *tar.Header, path string, fi os.FileInfo) (wo *tar.Header, err error) {
// convert whiteouts to AUFS format
if fi.Mode()&os.ModeCharDevice != 0 && hdr.Devmajor == 0 && hdr.Devminor == 0 {
// we just rename the file and make it normal
dir, filename := filepath.Split(hdr.Name)
hdr.Name = filepath.Join(dir, WhiteoutPrefix+filename)
hdr.Mode = 0600
hdr.Typeflag = tar.TypeReg
hdr.Size = 0
}
if fi.Mode()&os.ModeDir != 0 {
// convert opaque dirs to AUFS format by writing an empty file with the prefix
opaque, err := system.Lgetxattr(path, "trusted.overlay.opaque")
if err != nil {
return nil, err
}
if len(opaque) == 1 && opaque[0] == 'y' {
if hdr.Xattrs != nil {
delete(hdr.Xattrs, "trusted.overlay.opaque")
}
// create a header for the whiteout file
// it should inherit some properties from the parent, but be a regular file
wo = &tar.Header{
Typeflag: tar.TypeReg,
Mode: hdr.Mode & int64(os.ModePerm),
Name: filepath.Join(hdr.Name, WhiteoutOpaqueDir),
Size: 0,
Uid: hdr.Uid,
Uname: hdr.Uname,
Gid: hdr.Gid,
Gname: hdr.Gname,
AccessTime: hdr.AccessTime,
ChangeTime: hdr.ChangeTime,
}
}
}
return
}
func (overlayWhiteoutConverter) ConvertRead(hdr *tar.Header, path string) (bool, error) {
base := filepath.Base(path)
dir := filepath.Dir(path)
// if a directory is marked as opaque by the AUFS special file, we need to translate that to overlay
if base == WhiteoutOpaqueDir {
err := unix.Setxattr(dir, "trusted.overlay.opaque", []byte{'y'}, 0)
// don't write the file itself
return false, err
}
// if a file was deleted and we are using overlay, we need to create a character device
if strings.HasPrefix(base, WhiteoutPrefix) {
originalBase := base[len(WhiteoutPrefix):]
originalPath := filepath.Join(dir, originalBase)
if err := unix.Mknod(originalPath, unix.S_IFCHR, 0); err != nil {
return false, err
}
if err := os.Chown(originalPath, hdr.Uid, hdr.Gid); err != nil {
return false, err
}
// don't write the file itself
return false, nil
}
return true, nil
}
@@ -0,0 +1,7 @@
// +build !linux
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
func getWhiteoutConverter(format WhiteoutFormat) tarWhiteoutConverter {
return nil
}
+145
View File
@@ -0,0 +1,145 @@
// +build !windows
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"archive/tar"
"bufio"
"errors"
"fmt"
"os"
"path/filepath"
"syscall"
"github.com/ory/dockertest/v3/docker/pkg/idtools"
"github.com/ory/dockertest/v3/docker/pkg/system"
"golang.org/x/sys/unix"
)
// fixVolumePathPrefix does platform specific processing to ensure that if
// the path being passed in is not in a volume path format, convert it to one.
func fixVolumePathPrefix(srcPath string) string {
return srcPath
}
// getWalkRoot calculates the root path when performing a TarWithOptions.
// We use a separate function as this is platform specific. On Linux, we
// can't use filepath.Join(srcPath,include) because this will clean away
// a trailing "." or "/" which may be important.
func getWalkRoot(srcPath string, include string) string {
return srcPath + string(filepath.Separator) + include
}
// CanonicalTarNameForPath returns platform-specific filepath
// to canonical posix-style path for tar archival. p is relative
// path.
func CanonicalTarNameForPath(p string) (string, error) {
return p, nil // already unix-style
}
// chmodTarEntry is used to adjust the file permissions used in tar header based
// on the platform the archival is done.
func chmodTarEntry(perm os.FileMode) os.FileMode {
return perm // noop for unix as golang APIs provide perm bits correctly
}
func setHeaderForSpecialDevice(hdr *tar.Header, name string, stat interface{}) (err error) {
s, ok := stat.(*syscall.Stat_t)
if ok {
// Currently go does not fill in the major/minors
if s.Mode&unix.S_IFBLK != 0 ||
s.Mode&unix.S_IFCHR != 0 {
hdr.Devmajor = int64(unix.Major(uint64(s.Rdev))) // nolint: unconvert
hdr.Devminor = int64(unix.Minor(uint64(s.Rdev))) // nolint: unconvert
}
}
return
}
func getInodeFromStat(stat interface{}) (inode uint64, err error) {
s, ok := stat.(*syscall.Stat_t)
if ok {
inode = s.Ino
}
return
}
func getFileUIDGID(stat interface{}) (idtools.IDPair, error) {
s, ok := stat.(*syscall.Stat_t)
if !ok {
return idtools.IDPair{}, errors.New("cannot convert stat value to syscall.Stat_t")
}
return idtools.IDPair{UID: int(s.Uid), GID: int(s.Gid)}, nil
}
// handleTarTypeBlockCharFifo is an OS-specific helper function used by
// createTarFile to handle the following types of header: Block; Char; Fifo
func handleTarTypeBlockCharFifo(hdr *tar.Header, path string) error {
if runningInUserNS() {
// cannot create a device if running in user namespace
return nil
}
mode := uint32(hdr.Mode & 07777)
switch hdr.Typeflag {
case tar.TypeBlock:
mode |= unix.S_IFBLK
case tar.TypeChar:
mode |= unix.S_IFCHR
case tar.TypeFifo:
mode |= unix.S_IFIFO
}
return system.Mknod(path, mode, int(system.Mkdev(hdr.Devmajor, hdr.Devminor)))
}
func handleLChmod(hdr *tar.Header, path string, hdrInfo os.FileInfo) error {
if hdr.Typeflag == tar.TypeLink {
if fi, err := os.Lstat(hdr.Linkname); err == nil && (fi.Mode()&os.ModeSymlink == 0) {
if err := os.Chmod(path, hdrInfo.Mode()); err != nil {
return err
}
}
} else if hdr.Typeflag != tar.TypeSymlink {
if err := os.Chmod(path, hdrInfo.Mode()); err != nil {
return err
}
}
return nil
}
// runningInUserNS detects whether we are currently running in a user namespace.
// Copied from github.com/opencontainers/runc/libcontainer/system/linux.go
// Copied from github.com/lxc/lxd/shared/util.go
func runningInUserNS() bool {
file, err := os.Open("/proc/self/uid_map")
if err != nil {
// This kernel-provided file only exists if user namespaces are supported
return false
}
defer file.Close()
buf := bufio.NewReader(file)
l, _, err := buf.ReadLine()
if err != nil {
return false
}
line := string(l)
var a, b, c int64
fmt.Sscanf(line, "%d %d %d", &a, &b, &c)
/*
* We assume we are in the initial user namespace if we have a full
* range - 4294967295 uids starting at uid 0.
*/
if a == 0 && b == 0 && c == 4294967295 {
return false
}
return true
}
@@ -0,0 +1,77 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"archive/tar"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/ory/dockertest/v3/docker/pkg/idtools"
"github.com/ory/dockertest/v3/docker/pkg/longpath"
)
// fixVolumePathPrefix does platform specific processing to ensure that if
// the path being passed in is not in a volume path format, convert it to one.
func fixVolumePathPrefix(srcPath string) string {
return longpath.AddPrefix(srcPath)
}
// getWalkRoot calculates the root path when performing a TarWithOptions.
// We use a separate function as this is platform specific.
func getWalkRoot(srcPath string, include string) string {
return filepath.Join(srcPath, include)
}
// CanonicalTarNameForPath returns platform-specific filepath
// to canonical posix-style path for tar archival. p is relative
// path.
func CanonicalTarNameForPath(p string) (string, error) {
// windows: convert windows style relative path with backslashes
// into forward slashes. Since windows does not allow '/' or '\'
// in file names, it is mostly safe to replace however we must
// check just in case
if strings.Contains(p, "/") {
return "", fmt.Errorf("Windows path contains forward slash: %s", p)
}
return strings.Replace(p, string(os.PathSeparator), "/", -1), nil
}
// chmodTarEntry is used to adjust the file permissions used in tar header based
// on the platform the archival is done.
func chmodTarEntry(perm os.FileMode) os.FileMode {
//perm &= 0755 // this 0-ed out tar flags (like link, regular file, directory marker etc.)
permPart := perm & os.ModePerm
noPermPart := perm &^ os.ModePerm
// Add the x bit: make everything +x from windows
permPart |= 0111
permPart &= 0755
return noPermPart | permPart
}
func setHeaderForSpecialDevice(hdr *tar.Header, name string, stat interface{}) (err error) {
// do nothing. no notion of Rdev, Nlink in stat on Windows
return
}
func getInodeFromStat(stat interface{}) (inode uint64, err error) {
// do nothing. no notion of Inode in stat on Windows
return
}
// handleTarTypeBlockCharFifo is an OS-specific helper function used by
// createTarFile to handle the following types of header: Block; Char; Fifo
func handleTarTypeBlockCharFifo(hdr *tar.Header, path string) error {
return nil
}
func handleLChmod(hdr *tar.Header, path string, hdrInfo os.FileInfo) error {
return nil
}
func getFileUIDGID(stat interface{}) (idtools.IDPair, error) {
// no notion of file ownership mapping yet on Windows
return idtools.IDPair{0, 0}, nil
}
+441
View File
@@ -0,0 +1,441 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"archive/tar"
"bytes"
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"sort"
"strings"
"syscall"
"time"
"github.com/ory/dockertest/v3/docker/pkg/idtools"
"github.com/ory/dockertest/v3/docker/pkg/pools"
"github.com/ory/dockertest/v3/docker/pkg/system"
"github.com/sirupsen/logrus"
)
// ChangeType represents the change type.
type ChangeType int
const (
// ChangeModify represents the modify operation.
ChangeModify = iota
// ChangeAdd represents the add operation.
ChangeAdd
// ChangeDelete represents the delete operation.
ChangeDelete
)
func (c ChangeType) String() string {
switch c {
case ChangeModify:
return "C"
case ChangeAdd:
return "A"
case ChangeDelete:
return "D"
}
return ""
}
// Change represents a change, it wraps the change type and path.
// It describes changes of the files in the path respect to the
// parent layers. The change could be modify, add, delete.
// This is used for layer diff.
type Change struct {
Path string
Kind ChangeType
}
func (change *Change) String() string {
return fmt.Sprintf("%s %s", change.Kind, change.Path)
}
// for sort.Sort
type changesByPath []Change
func (c changesByPath) Less(i, j int) bool { return c[i].Path < c[j].Path }
func (c changesByPath) Len() int { return len(c) }
func (c changesByPath) Swap(i, j int) { c[j], c[i] = c[i], c[j] }
// Gnu tar and the go tar writer don't have sub-second mtime
// precision, which is problematic when we apply changes via tar
// files, we handle this by comparing for exact times, *or* same
// second count and either a or b having exactly 0 nanoseconds
func sameFsTime(a, b time.Time) bool {
return a == b ||
(a.Unix() == b.Unix() &&
(a.Nanosecond() == 0 || b.Nanosecond() == 0))
}
func sameFsTimeSpec(a, b syscall.Timespec) bool {
return a.Sec == b.Sec &&
(a.Nsec == b.Nsec || a.Nsec == 0 || b.Nsec == 0)
}
// Changes walks the path rw and determines changes for the files in the path,
// with respect to the parent layers
func Changes(layers []string, rw string) ([]Change, error) {
return changes(layers, rw, aufsDeletedFile, aufsMetadataSkip)
}
func aufsMetadataSkip(path string) (skip bool, err error) {
skip, err = filepath.Match(string(os.PathSeparator)+WhiteoutMetaPrefix+"*", path)
if err != nil {
skip = true
}
return
}
func aufsDeletedFile(root, path string, fi os.FileInfo) (string, error) {
f := filepath.Base(path)
// If there is a whiteout, then the file was removed
if strings.HasPrefix(f, WhiteoutPrefix) {
originalFile := f[len(WhiteoutPrefix):]
return filepath.Join(filepath.Dir(path), originalFile), nil
}
return "", nil
}
type skipChange func(string) (bool, error)
type deleteChange func(string, string, os.FileInfo) (string, error)
func changes(layers []string, rw string, dc deleteChange, sc skipChange) ([]Change, error) {
var (
changes []Change
changedDirs = make(map[string]struct{})
)
err := filepath.Walk(rw, func(path string, f os.FileInfo, err error) error {
if err != nil {
return err
}
// Rebase path
path, err = filepath.Rel(rw, path)
if err != nil {
return err
}
// As this runs on the daemon side, file paths are OS specific.
path = filepath.Join(string(os.PathSeparator), path)
// Skip root
if path == string(os.PathSeparator) {
return nil
}
if sc != nil {
if skip, err := sc(path); skip {
return err
}
}
change := Change{
Path: path,
}
deletedFile, err := dc(rw, path, f)
if err != nil {
return err
}
// Find out what kind of modification happened
if deletedFile != "" {
change.Path = deletedFile
change.Kind = ChangeDelete
} else {
// Otherwise, the file was added
change.Kind = ChangeAdd
// ...Unless it already existed in a top layer, in which case, it's a modification
for _, layer := range layers {
stat, err := os.Stat(filepath.Join(layer, path))
if err != nil && !os.IsNotExist(err) {
return err
}
if err == nil {
// The file existed in the top layer, so that's a modification
// However, if it's a directory, maybe it wasn't actually modified.
// If you modify /foo/bar/baz, then /foo will be part of the changed files only because it's the parent of bar
if stat.IsDir() && f.IsDir() {
if f.Size() == stat.Size() && f.Mode() == stat.Mode() && sameFsTime(f.ModTime(), stat.ModTime()) {
// Both directories are the same, don't record the change
return nil
}
}
change.Kind = ChangeModify
break
}
}
}
// If /foo/bar/file.txt is modified, then /foo/bar must be part of the changed files.
// This block is here to ensure the change is recorded even if the
// modify time, mode and size of the parent directory in the rw and ro layers are all equal.
// Check https://github.com/docker/docker/pull/13590 for details.
if f.IsDir() {
changedDirs[path] = struct{}{}
}
if change.Kind == ChangeAdd || change.Kind == ChangeDelete {
parent := filepath.Dir(path)
if _, ok := changedDirs[parent]; !ok && parent != "/" {
changes = append(changes, Change{Path: parent, Kind: ChangeModify})
changedDirs[parent] = struct{}{}
}
}
// Record change
changes = append(changes, change)
return nil
})
if err != nil && !os.IsNotExist(err) {
return nil, err
}
return changes, nil
}
// FileInfo describes the information of a file.
type FileInfo struct {
parent *FileInfo
name string
stat *system.StatT
children map[string]*FileInfo
capability []byte
added bool
}
// LookUp looks up the file information of a file.
func (info *FileInfo) LookUp(path string) *FileInfo {
// As this runs on the daemon side, file paths are OS specific.
parent := info
if path == string(os.PathSeparator) {
return info
}
pathElements := strings.Split(path, string(os.PathSeparator))
for _, elem := range pathElements {
if elem != "" {
child := parent.children[elem]
if child == nil {
return nil
}
parent = child
}
}
return parent
}
func (info *FileInfo) path() string {
if info.parent == nil {
// As this runs on the daemon side, file paths are OS specific.
return string(os.PathSeparator)
}
return filepath.Join(info.parent.path(), info.name)
}
func (info *FileInfo) addChanges(oldInfo *FileInfo, changes *[]Change) {
sizeAtEntry := len(*changes)
if oldInfo == nil {
// add
change := Change{
Path: info.path(),
Kind: ChangeAdd,
}
*changes = append(*changes, change)
info.added = true
}
// We make a copy so we can modify it to detect additions
// also, we only recurse on the old dir if the new info is a directory
// otherwise any previous delete/change is considered recursive
oldChildren := make(map[string]*FileInfo)
if oldInfo != nil && info.isDir() {
for k, v := range oldInfo.children {
oldChildren[k] = v
}
}
for name, newChild := range info.children {
oldChild := oldChildren[name]
if oldChild != nil {
// change?
oldStat := oldChild.stat
newStat := newChild.stat
// Note: We can't compare inode or ctime or blocksize here, because these change
// when copying a file into a container. However, that is not generally a problem
// because any content change will change mtime, and any status change should
// be visible when actually comparing the stat fields. The only time this
// breaks down is if some code intentionally hides a change by setting
// back mtime
if statDifferent(oldStat, newStat) ||
!bytes.Equal(oldChild.capability, newChild.capability) {
change := Change{
Path: newChild.path(),
Kind: ChangeModify,
}
*changes = append(*changes, change)
newChild.added = true
}
// Remove from copy so we can detect deletions
delete(oldChildren, name)
}
newChild.addChanges(oldChild, changes)
}
for _, oldChild := range oldChildren {
// delete
change := Change{
Path: oldChild.path(),
Kind: ChangeDelete,
}
*changes = append(*changes, change)
}
// If there were changes inside this directory, we need to add it, even if the directory
// itself wasn't changed. This is needed to properly save and restore filesystem permissions.
// As this runs on the daemon side, file paths are OS specific.
if len(*changes) > sizeAtEntry && info.isDir() && !info.added && info.path() != string(os.PathSeparator) {
change := Change{
Path: info.path(),
Kind: ChangeModify,
}
// Let's insert the directory entry before the recently added entries located inside this dir
*changes = append(*changes, change) // just to resize the slice, will be overwritten
copy((*changes)[sizeAtEntry+1:], (*changes)[sizeAtEntry:])
(*changes)[sizeAtEntry] = change
}
}
// Changes add changes to file information.
func (info *FileInfo) Changes(oldInfo *FileInfo) []Change {
var changes []Change
info.addChanges(oldInfo, &changes)
return changes
}
func newRootFileInfo() *FileInfo {
// As this runs on the daemon side, file paths are OS specific.
root := &FileInfo{
name: string(os.PathSeparator),
children: make(map[string]*FileInfo),
}
return root
}
// ChangesDirs compares two directories and generates an array of Change objects describing the changes.
// If oldDir is "", then all files in newDir will be Add-Changes.
func ChangesDirs(newDir, oldDir string) ([]Change, error) {
var (
oldRoot, newRoot *FileInfo
)
if oldDir == "" {
emptyDir, err := ioutil.TempDir("", "empty")
if err != nil {
return nil, err
}
defer os.Remove(emptyDir)
oldDir = emptyDir
}
oldRoot, newRoot, err := collectFileInfoForChanges(oldDir, newDir)
if err != nil {
return nil, err
}
return newRoot.Changes(oldRoot), nil
}
// ChangesSize calculates the size in bytes of the provided changes, based on newDir.
func ChangesSize(newDir string, changes []Change) int64 {
var (
size int64
sf = make(map[uint64]struct{})
)
for _, change := range changes {
if change.Kind == ChangeModify || change.Kind == ChangeAdd {
file := filepath.Join(newDir, change.Path)
fileInfo, err := os.Lstat(file)
if err != nil {
logrus.Errorf("Can not stat %q: %s", file, err)
continue
}
if fileInfo != nil && !fileInfo.IsDir() {
if hasHardlinks(fileInfo) {
inode := getIno(fileInfo)
if _, ok := sf[inode]; !ok {
size += fileInfo.Size()
sf[inode] = struct{}{}
}
} else {
size += fileInfo.Size()
}
}
}
}
return size
}
// ExportChanges produces an Archive from the provided changes, relative to dir.
func ExportChanges(dir string, changes []Change, uidMaps, gidMaps []idtools.IDMap) (io.ReadCloser, error) {
reader, writer := io.Pipe()
go func() {
ta := newTarAppender(idtools.NewIDMappingsFromMaps(uidMaps, gidMaps), writer, nil)
// this buffer is needed for the duration of this piped stream
defer pools.BufioWriter32KPool.Put(ta.Buffer)
sort.Sort(changesByPath(changes))
// In general we log errors here but ignore them because
// during e.g. a diff operation the container can continue
// mutating the filesystem and we can see transient errors
// from this
for _, change := range changes {
if change.Kind == ChangeDelete {
whiteOutDir := filepath.Dir(change.Path)
whiteOutBase := filepath.Base(change.Path)
whiteOut := filepath.Join(whiteOutDir, WhiteoutPrefix+whiteOutBase)
timestamp := time.Now()
hdr := &tar.Header{
Name: whiteOut[1:],
Size: 0,
ModTime: timestamp,
AccessTime: timestamp,
ChangeTime: timestamp,
}
if err := ta.TarWriter.WriteHeader(hdr); err != nil {
logrus.Debugf("Can't write whiteout header: %s", err)
}
} else {
path := filepath.Join(dir, change.Path)
if err := ta.addTarFile(path, change.Path[1:]); err != nil {
logrus.Debugf("Can't add file %s to tar: %s", path, err)
}
}
}
// Make sure to check the error on Close.
if err := ta.TarWriter.Close(); err != nil {
logrus.Debugf("Can't close layer: %s", err)
}
if err := writer.Close(); err != nil {
logrus.Debugf("failed close Changes writer: %s", err)
}
}()
return reader, nil
}
+313
View File
@@ -0,0 +1,313 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"bytes"
"fmt"
"os"
"path/filepath"
"sort"
"syscall"
"unsafe"
"github.com/ory/dockertest/v3/docker/pkg/system"
"golang.org/x/sys/unix"
)
// walker is used to implement collectFileInfoForChanges on linux. Where this
// method in general returns the entire contents of two directory trees, we
// optimize some FS calls out on linux. In particular, we take advantage of the
// fact that getdents(2) returns the inode of each file in the directory being
// walked, which, when walking two trees in parallel to generate a list of
// changes, can be used to prune subtrees without ever having to lstat(2) them
// directly. Eliminating stat calls in this way can save up to seconds on large
// images.
type walker struct {
dir1 string
dir2 string
root1 *FileInfo
root2 *FileInfo
}
// collectFileInfoForChanges returns a complete representation of the trees
// rooted at dir1 and dir2, with one important exception: any subtree or
// leaf where the inode and device numbers are an exact match between dir1
// and dir2 will be pruned from the results. This method is *only* to be used
// to generating a list of changes between the two directories, as it does not
// reflect the full contents.
func collectFileInfoForChanges(dir1, dir2 string) (*FileInfo, *FileInfo, error) {
w := &walker{
dir1: dir1,
dir2: dir2,
root1: newRootFileInfo(),
root2: newRootFileInfo(),
}
i1, err := os.Lstat(w.dir1)
if err != nil {
return nil, nil, err
}
i2, err := os.Lstat(w.dir2)
if err != nil {
return nil, nil, err
}
if err := w.walk("/", i1, i2); err != nil {
return nil, nil, err
}
return w.root1, w.root2, nil
}
// Given a FileInfo, its path info, and a reference to the root of the tree
// being constructed, register this file with the tree.
func walkchunk(path string, fi os.FileInfo, dir string, root *FileInfo) error {
if fi == nil {
return nil
}
parent := root.LookUp(filepath.Dir(path))
if parent == nil {
return fmt.Errorf("walkchunk: Unexpectedly no parent for %s", path)
}
info := &FileInfo{
name: filepath.Base(path),
children: make(map[string]*FileInfo),
parent: parent,
}
cpath := filepath.Join(dir, path)
stat, err := system.FromStatT(fi.Sys().(*syscall.Stat_t))
if err != nil {
return err
}
info.stat = stat
info.capability, _ = system.Lgetxattr(cpath, "security.capability") // lgetxattr(2): fs access
parent.children[info.name] = info
return nil
}
// Walk a subtree rooted at the same path in both trees being iterated. For
// example, /docker/overlay/1234/a/b/c/d and /docker/overlay/8888/a/b/c/d
func (w *walker) walk(path string, i1, i2 os.FileInfo) (err error) {
// Register these nodes with the return trees, unless we're still at the
// (already-created) roots:
if path != "/" {
if err := walkchunk(path, i1, w.dir1, w.root1); err != nil {
return err
}
if err := walkchunk(path, i2, w.dir2, w.root2); err != nil {
return err
}
}
is1Dir := i1 != nil && i1.IsDir()
is2Dir := i2 != nil && i2.IsDir()
sameDevice := false
if i1 != nil && i2 != nil {
si1 := i1.Sys().(*syscall.Stat_t)
si2 := i2.Sys().(*syscall.Stat_t)
if si1.Dev == si2.Dev {
sameDevice = true
}
}
// If these files are both non-existent, or leaves (non-dirs), we are done.
if !is1Dir && !is2Dir {
return nil
}
// Fetch the names of all the files contained in both directories being walked:
var names1, names2 []nameIno
if is1Dir {
names1, err = readdirnames(filepath.Join(w.dir1, path)) // getdents(2): fs access
if err != nil {
return err
}
}
if is2Dir {
names2, err = readdirnames(filepath.Join(w.dir2, path)) // getdents(2): fs access
if err != nil {
return err
}
}
// We have lists of the files contained in both parallel directories, sorted
// in the same order. Walk them in parallel, generating a unique merged list
// of all items present in either or both directories.
var names []string
ix1 := 0
ix2 := 0
for {
if ix1 >= len(names1) {
break
}
if ix2 >= len(names2) {
break
}
ni1 := names1[ix1]
ni2 := names2[ix2]
switch bytes.Compare([]byte(ni1.name), []byte(ni2.name)) {
case -1: // ni1 < ni2 -- advance ni1
// we will not encounter ni1 in names2
names = append(names, ni1.name)
ix1++
case 0: // ni1 == ni2
if ni1.ino != ni2.ino || !sameDevice {
names = append(names, ni1.name)
}
ix1++
ix2++
case 1: // ni1 > ni2 -- advance ni2
// we will not encounter ni2 in names1
names = append(names, ni2.name)
ix2++
}
}
for ix1 < len(names1) {
names = append(names, names1[ix1].name)
ix1++
}
for ix2 < len(names2) {
names = append(names, names2[ix2].name)
ix2++
}
// For each of the names present in either or both of the directories being
// iterated, stat the name under each root, and recurse the pair of them:
for _, name := range names {
fname := filepath.Join(path, name)
var cInfo1, cInfo2 os.FileInfo
if is1Dir {
cInfo1, err = os.Lstat(filepath.Join(w.dir1, fname)) // lstat(2): fs access
if err != nil && !os.IsNotExist(err) {
return err
}
}
if is2Dir {
cInfo2, err = os.Lstat(filepath.Join(w.dir2, fname)) // lstat(2): fs access
if err != nil && !os.IsNotExist(err) {
return err
}
}
if err = w.walk(fname, cInfo1, cInfo2); err != nil {
return err
}
}
return nil
}
// {name,inode} pairs used to support the early-pruning logic of the walker type
type nameIno struct {
name string
ino uint64
}
type nameInoSlice []nameIno
func (s nameInoSlice) Len() int { return len(s) }
func (s nameInoSlice) Swap(i, j int) { s[i], s[j] = s[j], s[i] }
func (s nameInoSlice) Less(i, j int) bool { return s[i].name < s[j].name }
// readdirnames is a hacked-apart version of the Go stdlib code, exposing inode
// numbers further up the stack when reading directory contents. Unlike
// os.Readdirnames, which returns a list of filenames, this function returns a
// list of {filename,inode} pairs.
func readdirnames(dirname string) (names []nameIno, err error) {
var (
size = 100
buf = make([]byte, 4096)
nbuf int
bufp int
nb int
)
f, err := os.Open(dirname)
if err != nil {
return nil, err
}
defer f.Close()
names = make([]nameIno, 0, size) // Empty with room to grow.
for {
// Refill the buffer if necessary
if bufp >= nbuf {
bufp = 0
nbuf, err = unix.ReadDirent(int(f.Fd()), buf) // getdents on linux
if nbuf < 0 {
nbuf = 0
}
if err != nil {
return nil, os.NewSyscallError("readdirent", err)
}
if nbuf <= 0 {
break // EOF
}
}
// Drain the buffer
nb, names = parseDirent(buf[bufp:nbuf], names)
bufp += nb
}
sl := nameInoSlice(names)
sort.Sort(sl)
return sl, nil
}
// parseDirent is a minor modification of unix.ParseDirent (linux version)
// which returns {name,inode} pairs instead of just names.
func parseDirent(buf []byte, names []nameIno) (consumed int, newnames []nameIno) {
origlen := len(buf)
for len(buf) > 0 {
dirent := (*unix.Dirent)(unsafe.Pointer(&buf[0]))
buf = buf[dirent.Reclen:]
if dirent.Ino == 0 { // File absent in directory.
continue
}
bytes := (*[10000]byte)(unsafe.Pointer(&dirent.Name[0]))
var name = string(bytes[0:clen(bytes[:])])
if name == "." || name == ".." { // Useless names
continue
}
names = append(names, nameIno{name, dirent.Ino})
}
return origlen - len(buf), names
}
func clen(n []byte) int {
for i := 0; i < len(n); i++ {
if n[i] == 0 {
return i
}
}
return len(n)
}
// OverlayChanges walks the path rw and determines changes for the files in the path,
// with respect to the parent layers
func OverlayChanges(layers []string, rw string) ([]Change, error) {
return changes(layers, rw, overlayDeletedFile, nil)
}
func overlayDeletedFile(root, path string, fi os.FileInfo) (string, error) {
if fi.Mode()&os.ModeCharDevice != 0 {
s := fi.Sys().(*syscall.Stat_t)
if unix.Major(uint64(s.Rdev)) == 0 && unix.Minor(uint64(s.Rdev)) == 0 { // nolint: unconvert
return path, nil
}
}
if fi.Mode()&os.ModeDir != 0 {
opaque, err := system.Lgetxattr(filepath.Join(root, path), "trusted.overlay.opaque")
if err != nil {
return "", err
}
if len(opaque) == 1 && opaque[0] == 'y' {
return path, nil
}
}
return "", nil
}
@@ -0,0 +1,97 @@
// +build !linux
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"fmt"
"os"
"path/filepath"
"runtime"
"strings"
"github.com/ory/dockertest/v3/docker/pkg/system"
)
func collectFileInfoForChanges(oldDir, newDir string) (*FileInfo, *FileInfo, error) {
var (
oldRoot, newRoot *FileInfo
err1, err2 error
errs = make(chan error, 2)
)
go func() {
oldRoot, err1 = collectFileInfo(oldDir)
errs <- err1
}()
go func() {
newRoot, err2 = collectFileInfo(newDir)
errs <- err2
}()
// block until both routines have returned
for i := 0; i < 2; i++ {
if err := <-errs; err != nil {
return nil, nil, err
}
}
return oldRoot, newRoot, nil
}
func collectFileInfo(sourceDir string) (*FileInfo, error) {
root := newRootFileInfo()
err := filepath.Walk(sourceDir, func(path string, f os.FileInfo, err error) error {
if err != nil {
return err
}
// Rebase path
relPath, err := filepath.Rel(sourceDir, path)
if err != nil {
return err
}
// As this runs on the daemon side, file paths are OS specific.
relPath = filepath.Join(string(os.PathSeparator), relPath)
// See https://github.com/golang/go/issues/9168 - bug in filepath.Join.
// Temporary workaround. If the returned path starts with two backslashes,
// trim it down to a single backslash. Only relevant on Windows.
if runtime.GOOS == "windows" {
if strings.HasPrefix(relPath, `\\`) {
relPath = relPath[1:]
}
}
if relPath == string(os.PathSeparator) {
return nil
}
parent := root.LookUp(filepath.Dir(relPath))
if parent == nil {
return fmt.Errorf("collectFileInfo: Unexpectedly no parent for %s", relPath)
}
info := &FileInfo{
name: filepath.Base(relPath),
children: make(map[string]*FileInfo),
parent: parent,
}
s, err := system.Lstat(path)
if err != nil {
return err
}
info.stat = s
info.capability, _ = system.Lgetxattr(path, "security.capability")
parent.children[info.name] = info
return nil
})
if err != nil {
return nil, err
}
return root, nil
}
+37
View File
@@ -0,0 +1,37 @@
// +build !windows
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"os"
"syscall"
"github.com/ory/dockertest/v3/docker/pkg/system"
"golang.org/x/sys/unix"
)
func statDifferent(oldStat *system.StatT, newStat *system.StatT) bool {
// Don't look at size for dirs, its not a good measure of change
if oldStat.Mode() != newStat.Mode() ||
oldStat.UID() != newStat.UID() ||
oldStat.GID() != newStat.GID() ||
oldStat.Rdev() != newStat.Rdev() ||
// Don't look at size for dirs, its not a good measure of change
(oldStat.Mode()&unix.S_IFDIR != unix.S_IFDIR &&
(!sameFsTimeSpec(oldStat.Mtim(), newStat.Mtim()) || (oldStat.Size() != newStat.Size()))) {
return true
}
return false
}
func (info *FileInfo) isDir() bool {
return info.parent == nil || info.stat.Mode()&unix.S_IFDIR != 0
}
func getIno(fi os.FileInfo) uint64 {
return fi.Sys().(*syscall.Stat_t).Ino
}
func hasHardlinks(fi os.FileInfo) bool {
return fi.Sys().(*syscall.Stat_t).Nlink > 1
}
@@ -0,0 +1,30 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"os"
"github.com/ory/dockertest/v3/docker/pkg/system"
)
func statDifferent(oldStat *system.StatT, newStat *system.StatT) bool {
// Don't look at size for dirs, its not a good measure of change
if oldStat.Mtim() != newStat.Mtim() ||
oldStat.Mode() != newStat.Mode() ||
oldStat.Size() != newStat.Size() && !oldStat.Mode().IsDir() {
return true
}
return false
}
func (info *FileInfo) isDir() bool {
return info.parent == nil || info.stat.Mode().IsDir()
}
func getIno(fi os.FileInfo) (inode uint64) {
return
}
func hasHardlinks(fi os.FileInfo) bool {
return false
}
+472
View File
@@ -0,0 +1,472 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"archive/tar"
"errors"
"io"
"io/ioutil"
"os"
"path/filepath"
"strings"
"github.com/ory/dockertest/v3/docker/pkg/system"
"github.com/sirupsen/logrus"
)
// Errors used or returned by this file.
var (
ErrNotDirectory = errors.New("not a directory")
ErrDirNotExists = errors.New("no such directory")
ErrCannotCopyDir = errors.New("cannot copy directory")
ErrInvalidCopySource = errors.New("invalid copy source content")
)
// PreserveTrailingDotOrSeparator returns the given cleaned path (after
// processing using any utility functions from the path or filepath stdlib
// packages) and appends a trailing `/.` or `/` if its corresponding original
// path (from before being processed by utility functions from the path or
// filepath stdlib packages) ends with a trailing `/.` or `/`. If the cleaned
// path already ends in a `.` path segment, then another is not added. If the
// clean path already ends in the separator, then another is not added.
func PreserveTrailingDotOrSeparator(cleanedPath string, originalPath string, sep byte) string {
// Ensure paths are in platform semantics
cleanedPath = strings.Replace(cleanedPath, "/", string(sep), -1)
originalPath = strings.Replace(originalPath, "/", string(sep), -1)
if !specifiesCurrentDir(cleanedPath) && specifiesCurrentDir(originalPath) {
if !hasTrailingPathSeparator(cleanedPath, sep) {
// Add a separator if it doesn't already end with one (a cleaned
// path would only end in a separator if it is the root).
cleanedPath += string(sep)
}
cleanedPath += "."
}
if !hasTrailingPathSeparator(cleanedPath, sep) && hasTrailingPathSeparator(originalPath, sep) {
cleanedPath += string(sep)
}
return cleanedPath
}
// assertsDirectory returns whether the given path is
// asserted to be a directory, i.e., the path ends with
// a trailing '/' or `/.`, assuming a path separator of `/`.
func assertsDirectory(path string, sep byte) bool {
return hasTrailingPathSeparator(path, sep) || specifiesCurrentDir(path)
}
// hasTrailingPathSeparator returns whether the given
// path ends with the system's path separator character.
func hasTrailingPathSeparator(path string, sep byte) bool {
return len(path) > 0 && path[len(path)-1] == sep
}
// specifiesCurrentDir returns whether the given path specifies
// a "current directory", i.e., the last path segment is `.`.
func specifiesCurrentDir(path string) bool {
return filepath.Base(path) == "."
}
// SplitPathDirEntry splits the given path between its directory name and its
// basename by first cleaning the path but preserves a trailing "." if the
// original path specified the current directory.
func SplitPathDirEntry(path string) (dir, base string) {
cleanedPath := filepath.Clean(filepath.FromSlash(path))
if specifiesCurrentDir(path) {
cleanedPath += string(os.PathSeparator) + "."
}
return filepath.Dir(cleanedPath), filepath.Base(cleanedPath)
}
// TarResource archives the resource described by the given CopyInfo to a Tar
// archive. A non-nil error is returned if sourcePath does not exist or is
// asserted to be a directory but exists as another type of file.
//
// This function acts as a convenient wrapper around TarWithOptions, which
// requires a directory as the source path. TarResource accepts either a
// directory or a file path and correctly sets the Tar options.
func TarResource(sourceInfo CopyInfo) (content io.ReadCloser, err error) {
return TarResourceRebase(sourceInfo.Path, sourceInfo.RebaseName)
}
// TarResourceRebase is like TarResource but renames the first path element of
// items in the resulting tar archive to match the given rebaseName if not "".
func TarResourceRebase(sourcePath, rebaseName string) (content io.ReadCloser, err error) {
sourcePath = normalizePath(sourcePath)
if _, err = os.Lstat(sourcePath); err != nil {
// Catches the case where the source does not exist or is not a
// directory if asserted to be a directory, as this also causes an
// error.
return
}
// Separate the source path between its directory and
// the entry in that directory which we are archiving.
sourceDir, sourceBase := SplitPathDirEntry(sourcePath)
opts := TarResourceRebaseOpts(sourceBase, rebaseName)
logrus.Debugf("copying %q from %q", sourceBase, sourceDir)
return TarWithOptions(sourceDir, opts)
}
// TarResourceRebaseOpts does not preform the Tar, but instead just creates the rebase
// parameters to be sent to TarWithOptions (the TarOptions struct)
func TarResourceRebaseOpts(sourceBase string, rebaseName string) *TarOptions {
filter := []string{sourceBase}
return &TarOptions{
Compression: Uncompressed,
IncludeFiles: filter,
IncludeSourceDir: true,
RebaseNames: map[string]string{
sourceBase: rebaseName,
},
}
}
// CopyInfo holds basic info about the source
// or destination path of a copy operation.
type CopyInfo struct {
Path string
Exists bool
IsDir bool
RebaseName string
}
// CopyInfoSourcePath stats the given path to create a CopyInfo
// struct representing that resource for the source of an archive copy
// operation. The given path should be an absolute local path. A source path
// has all symlinks evaluated that appear before the last path separator ("/"
// on Unix). As it is to be a copy source, the path must exist.
func CopyInfoSourcePath(path string, followLink bool) (CopyInfo, error) {
// normalize the file path and then evaluate the symbol link
// we will use the target file instead of the symbol link if
// followLink is set
path = normalizePath(path)
resolvedPath, rebaseName, err := ResolveHostSourcePath(path, followLink)
if err != nil {
return CopyInfo{}, err
}
stat, err := os.Lstat(resolvedPath)
if err != nil {
return CopyInfo{}, err
}
return CopyInfo{
Path: resolvedPath,
Exists: true,
IsDir: stat.IsDir(),
RebaseName: rebaseName,
}, nil
}
// CopyInfoDestinationPath stats the given path to create a CopyInfo
// struct representing that resource for the destination of an archive copy
// operation. The given path should be an absolute local path.
func CopyInfoDestinationPath(path string) (info CopyInfo, err error) {
maxSymlinkIter := 10 // filepath.EvalSymlinks uses 255, but 10 already seems like a lot.
path = normalizePath(path)
originalPath := path
stat, err := os.Lstat(path)
if err == nil && stat.Mode()&os.ModeSymlink == 0 {
// The path exists and is not a symlink.
return CopyInfo{
Path: path,
Exists: true,
IsDir: stat.IsDir(),
}, nil
}
// While the path is a symlink.
for n := 0; err == nil && stat.Mode()&os.ModeSymlink != 0; n++ {
if n > maxSymlinkIter {
// Don't follow symlinks more than this arbitrary number of times.
return CopyInfo{}, errors.New("too many symlinks in " + originalPath)
}
// The path is a symbolic link. We need to evaluate it so that the
// destination of the copy operation is the link target and not the
// link itself. This is notably different than CopyInfoSourcePath which
// only evaluates symlinks before the last appearing path separator.
// Also note that it is okay if the last path element is a broken
// symlink as the copy operation should create the target.
var linkTarget string
linkTarget, err = os.Readlink(path)
if err != nil {
return CopyInfo{}, err
}
if !system.IsAbs(linkTarget) {
// Join with the parent directory.
dstParent, _ := SplitPathDirEntry(path)
linkTarget = filepath.Join(dstParent, linkTarget)
}
path = linkTarget
stat, err = os.Lstat(path)
}
if err != nil {
// It's okay if the destination path doesn't exist. We can still
// continue the copy operation if the parent directory exists.
if !os.IsNotExist(err) {
return CopyInfo{}, err
}
// Ensure destination parent dir exists.
dstParent, _ := SplitPathDirEntry(path)
parentDirStat, err := os.Stat(dstParent)
if err != nil {
return CopyInfo{}, err
}
if !parentDirStat.IsDir() {
return CopyInfo{}, ErrNotDirectory
}
return CopyInfo{Path: path}, nil
}
// The path exists after resolving symlinks.
return CopyInfo{
Path: path,
Exists: true,
IsDir: stat.IsDir(),
}, nil
}
// PrepareArchiveCopy prepares the given srcContent archive, which should
// contain the archived resource described by srcInfo, to the destination
// described by dstInfo. Returns the possibly modified content archive along
// with the path to the destination directory which it should be extracted to.
func PrepareArchiveCopy(srcContent io.Reader, srcInfo, dstInfo CopyInfo) (dstDir string, content io.ReadCloser, err error) {
// Ensure in platform semantics
srcInfo.Path = normalizePath(srcInfo.Path)
dstInfo.Path = normalizePath(dstInfo.Path)
// Separate the destination path between its directory and base
// components in case the source archive contents need to be rebased.
dstDir, dstBase := SplitPathDirEntry(dstInfo.Path)
_, srcBase := SplitPathDirEntry(srcInfo.Path)
switch {
case dstInfo.Exists && dstInfo.IsDir:
// The destination exists as a directory. No alteration
// to srcContent is needed as its contents can be
// simply extracted to the destination directory.
return dstInfo.Path, ioutil.NopCloser(srcContent), nil
case dstInfo.Exists && srcInfo.IsDir:
// The destination exists as some type of file and the source
// content is a directory. This is an error condition since
// you cannot copy a directory to an existing file location.
return "", nil, ErrCannotCopyDir
case dstInfo.Exists:
// The destination exists as some type of file and the source content
// is also a file. The source content entry will have to be renamed to
// have a basename which matches the destination path's basename.
if len(srcInfo.RebaseName) != 0 {
srcBase = srcInfo.RebaseName
}
return dstDir, RebaseArchiveEntries(srcContent, srcBase, dstBase), nil
case srcInfo.IsDir:
// The destination does not exist and the source content is an archive
// of a directory. The archive should be extracted to the parent of
// the destination path instead, and when it is, the directory that is
// created as a result should take the name of the destination path.
// The source content entries will have to be renamed to have a
// basename which matches the destination path's basename.
if len(srcInfo.RebaseName) != 0 {
srcBase = srcInfo.RebaseName
}
return dstDir, RebaseArchiveEntries(srcContent, srcBase, dstBase), nil
case assertsDirectory(dstInfo.Path, os.PathSeparator):
// The destination does not exist and is asserted to be created as a
// directory, but the source content is not a directory. This is an
// error condition since you cannot create a directory from a file
// source.
return "", nil, ErrDirNotExists
default:
// The last remaining case is when the destination does not exist, is
// not asserted to be a directory, and the source content is not an
// archive of a directory. It this case, the destination file will need
// to be created when the archive is extracted and the source content
// entry will have to be renamed to have a basename which matches the
// destination path's basename.
if len(srcInfo.RebaseName) != 0 {
srcBase = srcInfo.RebaseName
}
return dstDir, RebaseArchiveEntries(srcContent, srcBase, dstBase), nil
}
}
// RebaseArchiveEntries rewrites the given srcContent archive replacing
// an occurrence of oldBase with newBase at the beginning of entry names.
func RebaseArchiveEntries(srcContent io.Reader, oldBase, newBase string) io.ReadCloser {
if oldBase == string(os.PathSeparator) {
// If oldBase specifies the root directory, use an empty string as
// oldBase instead so that newBase doesn't replace the path separator
// that all paths will start with.
oldBase = ""
}
rebased, w := io.Pipe()
go func() {
srcTar := tar.NewReader(srcContent)
rebasedTar := tar.NewWriter(w)
for {
hdr, err := srcTar.Next()
if err == io.EOF {
// Signals end of archive.
rebasedTar.Close()
w.Close()
return
}
if err != nil {
w.CloseWithError(err)
return
}
hdr.Name = strings.Replace(hdr.Name, oldBase, newBase, 1)
if hdr.Typeflag == tar.TypeLink {
hdr.Linkname = strings.Replace(hdr.Linkname, oldBase, newBase, 1)
}
if err = rebasedTar.WriteHeader(hdr); err != nil {
w.CloseWithError(err)
return
}
if _, err = io.Copy(rebasedTar, srcTar); err != nil {
w.CloseWithError(err)
return
}
}
}()
return rebased
}
// TODO @gupta-ak. These might have to be changed in the future to be
// continuity driver aware as well to support LCOW.
// CopyResource performs an archive copy from the given source path to the
// given destination path. The source path MUST exist and the destination
// path's parent directory must exist.
func CopyResource(srcPath, dstPath string, followLink bool) error {
var (
srcInfo CopyInfo
err error
)
// Ensure in platform semantics
srcPath = normalizePath(srcPath)
dstPath = normalizePath(dstPath)
// Clean the source and destination paths.
srcPath = PreserveTrailingDotOrSeparator(filepath.Clean(srcPath), srcPath, os.PathSeparator)
dstPath = PreserveTrailingDotOrSeparator(filepath.Clean(dstPath), dstPath, os.PathSeparator)
if srcInfo, err = CopyInfoSourcePath(srcPath, followLink); err != nil {
return err
}
content, err := TarResource(srcInfo)
if err != nil {
return err
}
defer content.Close()
return CopyTo(content, srcInfo, dstPath)
}
// CopyTo handles extracting the given content whose
// entries should be sourced from srcInfo to dstPath.
func CopyTo(content io.Reader, srcInfo CopyInfo, dstPath string) error {
// The destination path need not exist, but CopyInfoDestinationPath will
// ensure that at least the parent directory exists.
dstInfo, err := CopyInfoDestinationPath(normalizePath(dstPath))
if err != nil {
return err
}
dstDir, copyArchive, err := PrepareArchiveCopy(content, srcInfo, dstInfo)
if err != nil {
return err
}
defer copyArchive.Close()
options := &TarOptions{
NoLchown: true,
NoOverwriteDirNonDir: true,
}
return Untar(copyArchive, dstDir, options)
}
// ResolveHostSourcePath decides real path need to be copied with parameters such as
// whether to follow symbol link or not, if followLink is true, resolvedPath will return
// link target of any symbol link file, else it will only resolve symlink of directory
// but return symbol link file itself without resolving.
func ResolveHostSourcePath(path string, followLink bool) (resolvedPath, rebaseName string, err error) {
if followLink {
resolvedPath, err = filepath.EvalSymlinks(path)
if err != nil {
return
}
resolvedPath, rebaseName = GetRebaseName(path, resolvedPath)
} else {
dirPath, basePath := filepath.Split(path)
// if not follow symbol link, then resolve symbol link of parent dir
var resolvedDirPath string
resolvedDirPath, err = filepath.EvalSymlinks(dirPath)
if err != nil {
return
}
// resolvedDirPath will have been cleaned (no trailing path separators) so
// we can manually join it with the base path element.
resolvedPath = resolvedDirPath + string(filepath.Separator) + basePath
if hasTrailingPathSeparator(path, os.PathSeparator) &&
filepath.Base(path) != filepath.Base(resolvedPath) {
rebaseName = filepath.Base(path)
}
}
return resolvedPath, rebaseName, nil
}
// GetRebaseName normalizes and compares path and resolvedPath,
// return completed resolved path and rebased file name
func GetRebaseName(path, resolvedPath string) (string, string) {
// linkTarget will have been cleaned (no trailing path separators and dot) so
// we can manually join it with them
var rebaseName string
if specifiesCurrentDir(path) &&
!specifiesCurrentDir(resolvedPath) {
resolvedPath += string(filepath.Separator) + "."
}
if hasTrailingPathSeparator(path, os.PathSeparator) &&
!hasTrailingPathSeparator(resolvedPath, os.PathSeparator) {
resolvedPath += string(filepath.Separator)
}
if filepath.Base(path) != filepath.Base(resolvedPath) {
// In the case where the path had a trailing separator and a symlink
// evaluation has changed the last path component, we will need to
// rebase the name in the archive that is being copied to match the
// originally requested name.
rebaseName = filepath.Base(path)
}
return resolvedPath, rebaseName
}
+11
View File
@@ -0,0 +1,11 @@
// +build !windows
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"path/filepath"
)
func normalizePath(path string) string {
return filepath.ToSlash(path)
}
@@ -0,0 +1,9 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"path/filepath"
)
func normalizePath(path string) string {
return filepath.FromSlash(path)
}
+256
View File
@@ -0,0 +1,256 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"archive/tar"
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"runtime"
"strings"
"github.com/ory/dockertest/v3/docker/pkg/idtools"
"github.com/ory/dockertest/v3/docker/pkg/pools"
"github.com/ory/dockertest/v3/docker/pkg/system"
"github.com/sirupsen/logrus"
)
// UnpackLayer unpack `layer` to a `dest`. The stream `layer` can be
// compressed or uncompressed.
// Returns the size in bytes of the contents of the layer.
func UnpackLayer(dest string, layer io.Reader, options *TarOptions) (size int64, err error) {
tr := tar.NewReader(layer)
trBuf := pools.BufioReader32KPool.Get(tr)
defer pools.BufioReader32KPool.Put(trBuf)
var dirs []*tar.Header
unpackedPaths := make(map[string]struct{})
if options == nil {
options = &TarOptions{}
}
if options.ExcludePatterns == nil {
options.ExcludePatterns = []string{}
}
idMappings := idtools.NewIDMappingsFromMaps(options.UIDMaps, options.GIDMaps)
aufsTempdir := ""
aufsHardlinks := make(map[string]*tar.Header)
// Iterate through the files in the archive.
for {
hdr, err := tr.Next()
if err == io.EOF {
// end of tar archive
break
}
if err != nil {
return 0, err
}
size += hdr.Size
// Normalize name, for safety and for a simple is-root check
hdr.Name = filepath.Clean(hdr.Name)
// Windows does not support filenames with colons in them. Ignore
// these files. This is not a problem though (although it might
// appear that it is). Let's suppose a client is running docker pull.
// The daemon it points to is Windows. Would it make sense for the
// client to be doing a docker pull Ubuntu for example (which has files
// with colons in the name under /usr/share/man/man3)? No, absolutely
// not as it would really only make sense that they were pulling a
// Windows image. However, for development, it is necessary to be able
// to pull Linux images which are in the repository.
//
// TODO Windows. Once the registry is aware of what images are Windows-
// specific or Linux-specific, this warning should be changed to an error
// to cater for the situation where someone does manage to upload a Linux
// image but have it tagged as Windows inadvertently.
if runtime.GOOS == "windows" {
if strings.Contains(hdr.Name, ":") {
logrus.Warnf("Windows: Ignoring %s (is this a Linux image?)", hdr.Name)
continue
}
}
// Note as these operations are platform specific, so must the slash be.
if !strings.HasSuffix(hdr.Name, string(os.PathSeparator)) {
// Not the root directory, ensure that the parent directory exists.
// This happened in some tests where an image had a tarfile without any
// parent directories.
parent := filepath.Dir(hdr.Name)
parentPath := filepath.Join(dest, parent)
if _, err := os.Lstat(parentPath); err != nil && os.IsNotExist(err) {
err = system.MkdirAll(parentPath, 0600, "")
if err != nil {
return 0, err
}
}
}
// Skip AUFS metadata dirs
if strings.HasPrefix(hdr.Name, WhiteoutMetaPrefix) {
// Regular files inside /.wh..wh.plnk can be used as hardlink targets
// We don't want this directory, but we need the files in them so that
// such hardlinks can be resolved.
if strings.HasPrefix(hdr.Name, WhiteoutLinkDir) && hdr.Typeflag == tar.TypeReg {
basename := filepath.Base(hdr.Name)
aufsHardlinks[basename] = hdr
if aufsTempdir == "" {
if aufsTempdir, err = ioutil.TempDir("", "dockerplnk"); err != nil {
return 0, err
}
defer os.RemoveAll(aufsTempdir)
}
if err := createTarFile(filepath.Join(aufsTempdir, basename), dest, hdr, tr, true, nil, options.InUserNS); err != nil {
return 0, err
}
}
if hdr.Name != WhiteoutOpaqueDir {
continue
}
}
path := filepath.Join(dest, hdr.Name)
rel, err := filepath.Rel(dest, path)
if err != nil {
return 0, err
}
// Note as these operations are platform specific, so must the slash be.
if strings.HasPrefix(rel, ".."+string(os.PathSeparator)) {
return 0, breakoutError(fmt.Errorf("%q is outside of %q", hdr.Name, dest))
}
base := filepath.Base(path)
if strings.HasPrefix(base, WhiteoutPrefix) {
dir := filepath.Dir(path)
if base == WhiteoutOpaqueDir {
_, err := os.Lstat(dir)
if err != nil {
return 0, err
}
err = filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if err != nil {
if os.IsNotExist(err) {
err = nil // parent was deleted
}
return err
}
if path == dir {
return nil
}
if _, exists := unpackedPaths[path]; !exists {
err := os.RemoveAll(path)
return err
}
return nil
})
if err != nil {
return 0, err
}
} else {
originalBase := base[len(WhiteoutPrefix):]
originalPath := filepath.Join(dir, originalBase)
if err := os.RemoveAll(originalPath); err != nil {
return 0, err
}
}
} else {
// If path exits we almost always just want to remove and replace it.
// The only exception is when it is a directory *and* the file from
// the layer is also a directory. Then we want to merge them (i.e.
// just apply the metadata from the layer).
if fi, err := os.Lstat(path); err == nil {
if !(fi.IsDir() && hdr.Typeflag == tar.TypeDir) {
if err := os.RemoveAll(path); err != nil {
return 0, err
}
}
}
trBuf.Reset(tr)
srcData := io.Reader(trBuf)
srcHdr := hdr
// Hard links into /.wh..wh.plnk don't work, as we don't extract that directory, so
// we manually retarget these into the temporary files we extracted them into
if hdr.Typeflag == tar.TypeLink && strings.HasPrefix(filepath.Clean(hdr.Linkname), WhiteoutLinkDir) {
linkBasename := filepath.Base(hdr.Linkname)
srcHdr = aufsHardlinks[linkBasename]
if srcHdr == nil {
return 0, fmt.Errorf("Invalid aufs hardlink")
}
tmpFile, err := os.Open(filepath.Join(aufsTempdir, linkBasename))
if err != nil {
return 0, err
}
defer tmpFile.Close()
srcData = tmpFile
}
if err := remapIDs(idMappings, srcHdr); err != nil {
return 0, err
}
if err := createTarFile(path, dest, srcHdr, srcData, true, nil, options.InUserNS); err != nil {
return 0, err
}
// Directory mtimes must be handled at the end to avoid further
// file creation in them to modify the directory mtime
if hdr.Typeflag == tar.TypeDir {
dirs = append(dirs, hdr)
}
unpackedPaths[path] = struct{}{}
}
}
for _, hdr := range dirs {
path := filepath.Join(dest, hdr.Name)
if err := system.Chtimes(path, hdr.AccessTime, hdr.ModTime); err != nil {
return 0, err
}
}
return size, nil
}
// ApplyLayer parses a diff in the standard layer format from `layer`,
// and applies it to the directory `dest`. The stream `layer` can be
// compressed or uncompressed.
// Returns the size in bytes of the contents of the layer.
func ApplyLayer(dest string, layer io.Reader) (int64, error) {
return applyLayerHandler(dest, layer, &TarOptions{}, true)
}
// ApplyUncompressedLayer parses a diff in the standard layer format from
// `layer`, and applies it to the directory `dest`. The stream `layer`
// can only be uncompressed.
// Returns the size in bytes of the contents of the layer.
func ApplyUncompressedLayer(dest string, layer io.Reader, options *TarOptions) (int64, error) {
return applyLayerHandler(dest, layer, options, false)
}
// do the bulk load of ApplyLayer, but allow for not calling DecompressStream
func applyLayerHandler(dest string, layer io.Reader, options *TarOptions, decompress bool) (int64, error) {
dest = filepath.Clean(dest)
// We need to be able to set any perms
oldmask, err := system.Umask(0)
if err != nil {
return 0, err
}
defer system.Umask(oldmask) // ignore err, ErrNotSupportedPlatform
if decompress {
layer, err = DecompressStream(layer)
if err != nil {
return 0, err
}
}
return UnpackLayer(dest, layer, options)
}
+16
View File
@@ -0,0 +1,16 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"syscall"
"time"
)
func timeToTimespec(time time.Time) (ts syscall.Timespec) {
if time.IsZero() {
// Return UTIME_OMIT special value
ts.Sec = 0
ts.Nsec = ((1 << 30) - 2)
return
}
return syscall.NsecToTimespec(time.UnixNano())
}
@@ -0,0 +1,16 @@
// +build !linux
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"syscall"
"time"
)
func timeToTimespec(time time.Time) (ts syscall.Timespec) {
nsec := int64(0)
if !time.IsZero() {
nsec = time.UnixNano()
}
return syscall.NsecToTimespec(nsec)
}
+23
View File
@@ -0,0 +1,23 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
// Whiteouts are files with a special meaning for the layered filesystem.
// Docker uses AUFS whiteout files inside exported archives. In other
// filesystems these files are generated/handled on tar creation/extraction.
// WhiteoutPrefix prefix means file is a whiteout. If this is followed by a
// filename this means that file has been removed from the base layer.
const WhiteoutPrefix = ".wh."
// WhiteoutMetaPrefix prefix means whiteout has a special meaning and is not
// for removing an actual file. Normally these files are excluded from exported
// archives.
const WhiteoutMetaPrefix = WhiteoutPrefix + WhiteoutPrefix
// WhiteoutLinkDir is a directory AUFS uses for storing hardlink links to other
// layers. Normally these should not go into exported archives and all changed
// hardlinks should be copied to the top layer.
const WhiteoutLinkDir = WhiteoutMetaPrefix + "plnk"
// WhiteoutOpaqueDir file means directory has been made opaque - meaning
// readdir calls to this directory do not follow to lower layers.
const WhiteoutOpaqueDir = WhiteoutMetaPrefix + ".opq"
+59
View File
@@ -0,0 +1,59 @@
package archive // import "github.com/ory/dockertest/v3/docker/pkg/archive"
import (
"archive/tar"
"bytes"
"io"
)
// Generate generates a new archive from the content provided
// as input.
//
// `files` is a sequence of path/content pairs. A new file is
// added to the archive for each pair.
// If the last pair is incomplete, the file is created with an
// empty content. For example:
//
// Generate("foo.txt", "hello world", "emptyfile")
//
// The above call will return an archive with 2 files:
// * ./foo.txt with content "hello world"
// * ./empty with empty content
//
// FIXME: stream content instead of buffering
// FIXME: specify permissions and other archive metadata
func Generate(input ...string) (io.Reader, error) {
files := parseStringPairs(input...)
buf := new(bytes.Buffer)
tw := tar.NewWriter(buf)
for _, file := range files {
name, content := file[0], file[1]
hdr := &tar.Header{
Name: name,
Size: int64(len(content)),
}
if err := tw.WriteHeader(hdr); err != nil {
return nil, err
}
if _, err := tw.Write([]byte(content)); err != nil {
return nil, err
}
}
if err := tw.Close(); err != nil {
return nil, err
}
return buf, nil
}
func parseStringPairs(input ...string) (output [][2]string) {
output = make([][2]string, 0, len(input)/2+1)
for i := 0; i < len(input); i += 2 {
var pair [2]string
pair[0] = input[i]
if i+1 < len(input) {
pair[1] = input[i+1]
}
output = append(output, pair)
}
return
}
+298
View File
@@ -0,0 +1,298 @@
package fileutils // import "github.com/ory/dockertest/v3/docker/pkg/fileutils"
import (
"errors"
"fmt"
"io"
"os"
"path/filepath"
"regexp"
"strings"
"text/scanner"
"github.com/sirupsen/logrus"
)
// PatternMatcher allows checking paths agaist a list of patterns
type PatternMatcher struct {
patterns []*Pattern
exclusions bool
}
// NewPatternMatcher creates a new matcher object for specific patterns that can
// be used later to match against patterns against paths
func NewPatternMatcher(patterns []string) (*PatternMatcher, error) {
pm := &PatternMatcher{
patterns: make([]*Pattern, 0, len(patterns)),
}
for _, p := range patterns {
// Eliminate leading and trailing whitespace.
p = strings.TrimSpace(p)
if p == "" {
continue
}
p = filepath.Clean(p)
newp := &Pattern{}
if p[0] == '!' {
if len(p) == 1 {
return nil, errors.New("illegal exclusion pattern: \"!\"")
}
newp.exclusion = true
p = p[1:]
pm.exclusions = true
}
// Do some syntax checking on the pattern.
// filepath's Match() has some really weird rules that are inconsistent
// so instead of trying to dup their logic, just call Match() for its
// error state and if there is an error in the pattern return it.
// If this becomes an issue we can remove this since its really only
// needed in the error (syntax) case - which isn't really critical.
if _, err := filepath.Match(p, "."); err != nil {
return nil, err
}
newp.cleanedPattern = p
newp.dirs = strings.Split(p, string(os.PathSeparator))
pm.patterns = append(pm.patterns, newp)
}
return pm, nil
}
// Matches matches path against all the patterns. Matches is not safe to be
// called concurrently
func (pm *PatternMatcher) Matches(file string) (bool, error) {
matched := false
file = filepath.FromSlash(file)
parentPath := filepath.Dir(file)
parentPathDirs := strings.Split(parentPath, string(os.PathSeparator))
for _, pattern := range pm.patterns {
negative := false
if pattern.exclusion {
negative = true
}
match, err := pattern.match(file)
if err != nil {
return false, err
}
if !match && parentPath != "." {
// Check to see if the pattern matches one of our parent dirs.
if len(pattern.dirs) <= len(parentPathDirs) {
match, _ = pattern.match(strings.Join(parentPathDirs[:len(pattern.dirs)], string(os.PathSeparator)))
}
}
if match {
matched = !negative
}
}
if matched {
logrus.Debugf("Skipping excluded path: %s", file)
}
return matched, nil
}
// Exclusions returns true if any of the patterns define exclusions
func (pm *PatternMatcher) Exclusions() bool {
return pm.exclusions
}
// Patterns returns array of active patterns
func (pm *PatternMatcher) Patterns() []*Pattern {
return pm.patterns
}
// Pattern defines a single regexp used used to filter file paths.
type Pattern struct {
cleanedPattern string
dirs []string
regexp *regexp.Regexp
exclusion bool
}
func (p *Pattern) String() string {
return p.cleanedPattern
}
// Exclusion returns true if this pattern defines exclusion
func (p *Pattern) Exclusion() bool {
return p.exclusion
}
func (p *Pattern) match(path string) (bool, error) {
if p.regexp == nil {
if err := p.compile(); err != nil {
return false, filepath.ErrBadPattern
}
}
b := p.regexp.MatchString(path)
return b, nil
}
func (p *Pattern) compile() error {
regStr := "^"
pattern := p.cleanedPattern
// Go through the pattern and convert it to a regexp.
// We use a scanner so we can support utf-8 chars.
var scan scanner.Scanner
scan.Init(strings.NewReader(pattern))
sl := string(os.PathSeparator)
escSL := sl
if sl == `\` {
escSL += `\`
}
for scan.Peek() != scanner.EOF {
ch := scan.Next()
if ch == '*' {
if scan.Peek() == '*' {
// is some flavor of "**"
scan.Next()
// Treat **/ as ** so eat the "/"
if string(scan.Peek()) == sl {
scan.Next()
}
if scan.Peek() == scanner.EOF {
// is "**EOF" - to align with .gitignore just accept all
regStr += ".*"
} else {
// is "**"
// Note that this allows for any # of /'s (even 0) because
// the .* will eat everything, even /'s
regStr += "(.*" + escSL + ")?"
}
} else {
// is "*" so map it to anything but "/"
regStr += "[^" + escSL + "]*"
}
} else if ch == '?' {
// "?" is any char except "/"
regStr += "[^" + escSL + "]"
} else if ch == '.' || ch == '$' {
// Escape some regexp special chars that have no meaning
// in golang's filepath.Match
regStr += `\` + string(ch)
} else if ch == '\\' {
// escape next char. Note that a trailing \ in the pattern
// will be left alone (but need to escape it)
if sl == `\` {
// On windows map "\" to "\\", meaning an escaped backslash,
// and then just continue because filepath.Match on
// Windows doesn't allow escaping at all
regStr += escSL
continue
}
if scan.Peek() != scanner.EOF {
regStr += `\` + string(scan.Next())
} else {
regStr += `\`
}
} else {
regStr += string(ch)
}
}
regStr += "$"
re, err := regexp.Compile(regStr)
if err != nil {
return err
}
p.regexp = re
return nil
}
// Matches returns true if file matches any of the patterns
// and isn't excluded by any of the subsequent patterns.
func Matches(file string, patterns []string) (bool, error) {
pm, err := NewPatternMatcher(patterns)
if err != nil {
return false, err
}
file = filepath.Clean(file)
if file == "." {
// Don't let them exclude everything, kind of silly.
return false, nil
}
return pm.Matches(file)
}
// CopyFile copies from src to dst until either EOF is reached
// on src or an error occurs. It verifies src exists and removes
// the dst if it exists.
func CopyFile(src, dst string) (int64, error) {
cleanSrc := filepath.Clean(src)
cleanDst := filepath.Clean(dst)
if cleanSrc == cleanDst {
return 0, nil
}
sf, err := os.Open(cleanSrc)
if err != nil {
return 0, err
}
defer sf.Close()
if err := os.Remove(cleanDst); err != nil && !os.IsNotExist(err) {
return 0, err
}
df, err := os.Create(cleanDst)
if err != nil {
return 0, err
}
defer df.Close()
return io.Copy(df, sf)
}
// ReadSymlinkedDirectory returns the target directory of a symlink.
// The target of the symbolic link may not be a file.
func ReadSymlinkedDirectory(path string) (string, error) {
var realPath string
var err error
if realPath, err = filepath.Abs(path); err != nil {
return "", fmt.Errorf("unable to get absolute path for %s: %s", path, err)
}
if realPath, err = filepath.EvalSymlinks(realPath); err != nil {
return "", fmt.Errorf("failed to canonicalise path for %s: %s", path, err)
}
realPathInfo, err := os.Stat(realPath)
if err != nil {
return "", fmt.Errorf("failed to stat target '%s' of '%s': %s", realPath, path, err)
}
if !realPathInfo.Mode().IsDir() {
return "", fmt.Errorf("canonical path points to a file '%s'", realPath)
}
return realPath, nil
}
// CreateIfNotExists creates a file or a directory only if it does not already exist.
func CreateIfNotExists(path string, isDir bool) error {
if _, err := os.Stat(path); err != nil {
if os.IsNotExist(err) {
if isDir {
return os.MkdirAll(path, 0755)
}
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
return err
}
f, err := os.OpenFile(path, os.O_CREATE, 0755)
if err != nil {
return err
}
f.Close()
}
}
return nil
}
@@ -0,0 +1,27 @@
package fileutils // import "github.com/ory/dockertest/v3/docker/pkg/fileutils"
import (
"os"
"os/exec"
"strconv"
"strings"
)
// GetTotalUsedFds returns the number of used File Descriptors by
// executing `lsof -p PID`
func GetTotalUsedFds() int {
pid := os.Getpid()
cmd := exec.Command("lsof", "-p", strconv.Itoa(pid))
output, err := cmd.CombinedOutput()
if err != nil {
return -1
}
outputStr := strings.TrimSpace(string(output))
fds := strings.Split(outputStr, "\n")
return len(fds) - 1
}
@@ -0,0 +1,22 @@
// +build linux freebsd
package fileutils // import "github.com/ory/dockertest/v3/docker/pkg/fileutils"
import (
"fmt"
"io/ioutil"
"os"
"github.com/sirupsen/logrus"
)
// GetTotalUsedFds Returns the number of used File Descriptors by
// reading it via /proc filesystem.
func GetTotalUsedFds() int {
if fds, err := ioutil.ReadDir(fmt.Sprintf("/proc/%d/fd", os.Getpid())); err != nil {
logrus.Errorf("Error opening /proc/%d/fd: %s", os.Getpid(), err)
} else {
return len(fds)
}
return -1
}
@@ -0,0 +1,7 @@
package fileutils // import "github.com/ory/dockertest/v3/docker/pkg/fileutils"
// GetTotalUsedFds Returns the number of used File Descriptors. Not supported
// on Windows.
func GetTotalUsedFds() int {
return -1
}
@@ -0,0 +1,21 @@
package homedir // import "github.com/ory/dockertest/v3/docker/pkg/homedir"
import (
"os"
"github.com/ory/dockertest/v3/docker/pkg/idtools"
)
// GetStatic returns the home directory for the current user without calling
// os/user.Current(). This is useful for static-linked binary on glibc-based
// system, because a call to os/user.Current() in a static binary leads to
// segfault due to a glibc issue that won't be fixed in a short term.
// (#29344, golang/go#13470, https://sourceware.org/bugzilla/show_bug.cgi?id=19341)
func GetStatic() (string, error) {
uid := os.Getuid()
usr, err := idtools.LookupUID(uid)
if err != nil {
return "", err
}
return usr.Home, nil
}
@@ -0,0 +1,13 @@
// +build !linux
package homedir // import "github.com/ory/dockertest/v3/docker/pkg/homedir"
import (
"errors"
)
// GetStatic is not needed for non-linux systems.
// (Precisely, it is needed only for glibc-based linux systems.)
func GetStatic() (string, error) {
return "", errors.New("homedir.GetStatic() is not supported on this system")
}
+34
View File
@@ -0,0 +1,34 @@
// +build !windows
package homedir // import "github.com/ory/dockertest/v3/docker/pkg/homedir"
import (
"os"
"github.com/opencontainers/runc/libcontainer/user"
)
// Key returns the env var name for the user's home dir based on
// the platform being run on
func Key() string {
return "HOME"
}
// Get returns the home directory of the current user with the help of
// environment variables depending on the target operating system.
// Returned path should be used with "path/filepath" to form new paths.
func Get() string {
home := os.Getenv(Key())
if home == "" {
if u, err := user.CurrentUser(); err == nil {
return u.Home
}
}
return home
}
// GetShortcutString returns the string that is shortcut to user's home directory
// in the native shell of the platform running on.
func GetShortcutString() string {
return "~"
}
@@ -0,0 +1,24 @@
package homedir // import "github.com/ory/dockertest/v3/docker/pkg/homedir"
import (
"os"
)
// Key returns the env var name for the user's home dir based on
// the platform being run on
func Key() string {
return "USERPROFILE"
}
// Get returns the home directory of the current user with the help of
// environment variables depending on the target operating system.
// Returned path should be used with "path/filepath" to form new paths.
func Get() string {
return os.Getenv(Key())
}
// GetShortcutString returns the string that is shortcut to user's home directory
// in the native shell of the platform running on.
func GetShortcutString() string {
return "%USERPROFILE%" // be careful while using in format functions
}
+266
View File
@@ -0,0 +1,266 @@
package idtools // import "github.com/ory/dockertest/v3/docker/pkg/idtools"
import (
"bufio"
"fmt"
"os"
"sort"
"strconv"
"strings"
)
// IDMap contains a single entry for user namespace range remapping. An array
// of IDMap entries represents the structure that will be provided to the Linux
// kernel for creating a user namespace.
type IDMap struct {
ContainerID int `json:"container_id"`
HostID int `json:"host_id"`
Size int `json:"size"`
}
type subIDRange struct {
Start int
Length int
}
type ranges []subIDRange
func (e ranges) Len() int { return len(e) }
func (e ranges) Swap(i, j int) { e[i], e[j] = e[j], e[i] }
func (e ranges) Less(i, j int) bool { return e[i].Start < e[j].Start }
const (
subuidFileName string = "/etc/subuid"
subgidFileName string = "/etc/subgid"
)
// MkdirAllAndChown creates a directory (include any along the path) and then modifies
// ownership to the requested uid/gid. If the directory already exists, this
// function will still change ownership to the requested uid/gid pair.
func MkdirAllAndChown(path string, mode os.FileMode, owner IDPair) error {
return mkdirAs(path, mode, owner.UID, owner.GID, true, true)
}
// MkdirAndChown creates a directory and then modifies ownership to the requested uid/gid.
// If the directory already exists, this function still changes ownership.
// Note that unlike os.Mkdir(), this function does not return IsExist error
// in case path already exists.
func MkdirAndChown(path string, mode os.FileMode, owner IDPair) error {
return mkdirAs(path, mode, owner.UID, owner.GID, false, true)
}
// MkdirAllAndChownNew creates a directory (include any along the path) and then modifies
// ownership ONLY of newly created directories to the requested uid/gid. If the
// directories along the path exist, no change of ownership will be performed
func MkdirAllAndChownNew(path string, mode os.FileMode, owner IDPair) error {
return mkdirAs(path, mode, owner.UID, owner.GID, true, false)
}
// GetRootUIDGID retrieves the remapped root uid/gid pair from the set of maps.
// If the maps are empty, then the root uid/gid will default to "real" 0/0
func GetRootUIDGID(uidMap, gidMap []IDMap) (int, int, error) {
uid, err := toHost(0, uidMap)
if err != nil {
return -1, -1, err
}
gid, err := toHost(0, gidMap)
if err != nil {
return -1, -1, err
}
return uid, gid, nil
}
// toContainer takes an id mapping, and uses it to translate a
// host ID to the remapped ID. If no map is provided, then the translation
// assumes a 1-to-1 mapping and returns the passed in id
func toContainer(hostID int, idMap []IDMap) (int, error) {
if idMap == nil {
return hostID, nil
}
for _, m := range idMap {
if (hostID >= m.HostID) && (hostID <= (m.HostID + m.Size - 1)) {
contID := m.ContainerID + (hostID - m.HostID)
return contID, nil
}
}
return -1, fmt.Errorf("Host ID %d cannot be mapped to a container ID", hostID)
}
// toHost takes an id mapping and a remapped ID, and translates the
// ID to the mapped host ID. If no map is provided, then the translation
// assumes a 1-to-1 mapping and returns the passed in id #
func toHost(contID int, idMap []IDMap) (int, error) {
if idMap == nil {
return contID, nil
}
for _, m := range idMap {
if (contID >= m.ContainerID) && (contID <= (m.ContainerID + m.Size - 1)) {
hostID := m.HostID + (contID - m.ContainerID)
return hostID, nil
}
}
return -1, fmt.Errorf("Container ID %d cannot be mapped to a host ID", contID)
}
// IDPair is a UID and GID pair
type IDPair struct {
UID int
GID int
}
// IDMappings contains a mappings of UIDs and GIDs
type IDMappings struct {
uids []IDMap
gids []IDMap
}
// NewIDMappings takes a requested user and group name and
// using the data from /etc/sub{uid,gid} ranges, creates the
// proper uid and gid remapping ranges for that user/group pair
func NewIDMappings(username, groupname string) (*IDMappings, error) {
subuidRanges, err := parseSubuid(username)
if err != nil {
return nil, err
}
subgidRanges, err := parseSubgid(groupname)
if err != nil {
return nil, err
}
if len(subuidRanges) == 0 {
return nil, fmt.Errorf("No subuid ranges found for user %q", username)
}
if len(subgidRanges) == 0 {
return nil, fmt.Errorf("No subgid ranges found for group %q", groupname)
}
return &IDMappings{
uids: createIDMap(subuidRanges),
gids: createIDMap(subgidRanges),
}, nil
}
// NewIDMappingsFromMaps creates a new mapping from two slices
// Deprecated: this is a temporary shim while transitioning to IDMapping
func NewIDMappingsFromMaps(uids []IDMap, gids []IDMap) *IDMappings {
return &IDMappings{uids: uids, gids: gids}
}
// RootPair returns a uid and gid pair for the root user. The error is ignored
// because a root user always exists, and the defaults are correct when the uid
// and gid maps are empty.
func (i *IDMappings) RootPair() IDPair {
uid, gid, _ := GetRootUIDGID(i.uids, i.gids)
return IDPair{UID: uid, GID: gid}
}
// ToHost returns the host UID and GID for the container uid, gid.
// Remapping is only performed if the ids aren't already the remapped root ids
func (i *IDMappings) ToHost(pair IDPair) (IDPair, error) {
var err error
target := i.RootPair()
if pair.UID != target.UID {
target.UID, err = toHost(pair.UID, i.uids)
if err != nil {
return target, err
}
}
if pair.GID != target.GID {
target.GID, err = toHost(pair.GID, i.gids)
}
return target, err
}
// ToContainer returns the container UID and GID for the host uid and gid
func (i *IDMappings) ToContainer(pair IDPair) (int, int, error) {
uid, err := toContainer(pair.UID, i.uids)
if err != nil {
return -1, -1, err
}
gid, err := toContainer(pair.GID, i.gids)
return uid, gid, err
}
// Empty returns true if there are no id mappings
func (i *IDMappings) Empty() bool {
return len(i.uids) == 0 && len(i.gids) == 0
}
// UIDs return the UID mapping
// TODO: remove this once everything has been refactored to use pairs
func (i *IDMappings) UIDs() []IDMap {
return i.uids
}
// GIDs return the UID mapping
// TODO: remove this once everything has been refactored to use pairs
func (i *IDMappings) GIDs() []IDMap {
return i.gids
}
func createIDMap(subidRanges ranges) []IDMap {
idMap := []IDMap{}
// sort the ranges by lowest ID first
sort.Sort(subidRanges)
containerID := 0
for _, idrange := range subidRanges {
idMap = append(idMap, IDMap{
ContainerID: containerID,
HostID: idrange.Start,
Size: idrange.Length,
})
containerID = containerID + idrange.Length
}
return idMap
}
func parseSubuid(username string) (ranges, error) {
return parseSubidFile(subuidFileName, username)
}
func parseSubgid(username string) (ranges, error) {
return parseSubidFile(subgidFileName, username)
}
// parseSubidFile will read the appropriate file (/etc/subuid or /etc/subgid)
// and return all found ranges for a specified username. If the special value
// "ALL" is supplied for username, then all ranges in the file will be returned
func parseSubidFile(path, username string) (ranges, error) {
var rangeList ranges
subidFile, err := os.Open(path)
if err != nil {
return rangeList, err
}
defer subidFile.Close()
s := bufio.NewScanner(subidFile)
for s.Scan() {
if err := s.Err(); err != nil {
return rangeList, err
}
text := strings.TrimSpace(s.Text())
if text == "" || strings.HasPrefix(text, "#") {
continue
}
parts := strings.Split(text, ":")
if len(parts) != 3 {
return rangeList, fmt.Errorf("Cannot parse subuid/gid information: Format not correct for %s file", path)
}
if parts[0] == username || username == "ALL" {
startid, err := strconv.Atoi(parts[1])
if err != nil {
return rangeList, fmt.Errorf("String to int conversion failed during subuid/gid parsing of %s: %v", path, err)
}
length, err := strconv.Atoi(parts[2])
if err != nil {
return rangeList, fmt.Errorf("String to int conversion failed during subuid/gid parsing of %s: %v", path, err)
}
rangeList = append(rangeList, subIDRange{startid, length})
}
}
return rangeList, nil
}
+230
View File
@@ -0,0 +1,230 @@
// +build !windows
package idtools // import "github.com/ory/dockertest/v3/docker/pkg/idtools"
import (
"bytes"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"sync"
"syscall"
"github.com/ory/dockertest/v3/docker/pkg/system"
"github.com/opencontainers/runc/libcontainer/user"
)
var (
entOnce sync.Once
getentCmd string
)
func mkdirAs(path string, mode os.FileMode, ownerUID, ownerGID int, mkAll, chownExisting bool) error {
// make an array containing the original path asked for, plus (for mkAll == true)
// all path components leading up to the complete path that don't exist before we MkdirAll
// so that we can chown all of them properly at the end. If chownExisting is false, we won't
// chown the full directory path if it exists
var paths []string
stat, err := system.Stat(path)
if err == nil {
if !stat.IsDir() {
return &os.PathError{Op: "mkdir", Path: path, Err: syscall.ENOTDIR}
}
if !chownExisting {
return nil
}
// short-circuit--we were called with an existing directory and chown was requested
return lazyChown(path, ownerUID, ownerGID, stat)
}
if os.IsNotExist(err) {
paths = []string{path}
}
if mkAll {
// walk back to "/" looking for directories which do not exist
// and add them to the paths array for chown after creation
dirPath := path
for {
dirPath = filepath.Dir(dirPath)
if dirPath == "/" {
break
}
if _, err := os.Stat(dirPath); err != nil && os.IsNotExist(err) {
paths = append(paths, dirPath)
}
}
if err := system.MkdirAll(path, mode, ""); err != nil {
return err
}
} else {
if err := os.Mkdir(path, mode); err != nil && !os.IsExist(err) {
return err
}
}
// even if it existed, we will chown the requested path + any subpaths that
// didn't exist when we called MkdirAll
for _, pathComponent := range paths {
if err := lazyChown(pathComponent, ownerUID, ownerGID, nil); err != nil {
return err
}
}
return nil
}
// CanAccess takes a valid (existing) directory and a uid, gid pair and determines
// if that uid, gid pair has access (execute bit) to the directory
func CanAccess(path string, pair IDPair) bool {
statInfo, err := system.Stat(path)
if err != nil {
return false
}
fileMode := os.FileMode(statInfo.Mode())
permBits := fileMode.Perm()
return accessible(statInfo.UID() == uint32(pair.UID),
statInfo.GID() == uint32(pair.GID), permBits)
}
func accessible(isOwner, isGroup bool, perms os.FileMode) bool {
if isOwner && (perms&0100 == 0100) {
return true
}
if isGroup && (perms&0010 == 0010) {
return true
}
if perms&0001 == 0001 {
return true
}
return false
}
// LookupUser uses traditional local system files lookup (from libcontainer/user) on a username,
// followed by a call to `getent` for supporting host configured non-files passwd and group dbs
func LookupUser(username string) (user.User, error) {
// first try a local system files lookup using existing capabilities
usr, err := user.LookupUser(username)
if err == nil {
return usr, nil
}
// local files lookup failed; attempt to call `getent` to query configured passwd dbs
usr, err = getentUser(fmt.Sprintf("%s %s", "passwd", username))
if err != nil {
return user.User{}, err
}
return usr, nil
}
// LookupUID uses traditional local system files lookup (from libcontainer/user) on a uid,
// followed by a call to `getent` for supporting host configured non-files passwd and group dbs
func LookupUID(uid int) (user.User, error) {
// first try a local system files lookup using existing capabilities
usr, err := user.LookupUid(uid)
if err == nil {
return usr, nil
}
// local files lookup failed; attempt to call `getent` to query configured passwd dbs
return getentUser(fmt.Sprintf("%s %d", "passwd", uid))
}
func getentUser(args string) (user.User, error) {
reader, err := callGetent(args)
if err != nil {
return user.User{}, err
}
users, err := user.ParsePasswd(reader)
if err != nil {
return user.User{}, err
}
if len(users) == 0 {
return user.User{}, fmt.Errorf("getent failed to find passwd entry for %q", strings.Split(args, " ")[1])
}
return users[0], nil
}
// LookupGroup uses traditional local system files lookup (from libcontainer/user) on a group name,
// followed by a call to `getent` for supporting host configured non-files passwd and group dbs
func LookupGroup(groupname string) (user.Group, error) {
// first try a local system files lookup using existing capabilities
group, err := user.LookupGroup(groupname)
if err == nil {
return group, nil
}
// local files lookup failed; attempt to call `getent` to query configured group dbs
return getentGroup(fmt.Sprintf("%s %s", "group", groupname))
}
// LookupGID uses traditional local system files lookup (from libcontainer/user) on a group ID,
// followed by a call to `getent` for supporting host configured non-files passwd and group dbs
func LookupGID(gid int) (user.Group, error) {
// first try a local system files lookup using existing capabilities
group, err := user.LookupGid(gid)
if err == nil {
return group, nil
}
// local files lookup failed; attempt to call `getent` to query configured group dbs
return getentGroup(fmt.Sprintf("%s %d", "group", gid))
}
func getentGroup(args string) (user.Group, error) {
reader, err := callGetent(args)
if err != nil {
return user.Group{}, err
}
groups, err := user.ParseGroup(reader)
if err != nil {
return user.Group{}, err
}
if len(groups) == 0 {
return user.Group{}, fmt.Errorf("getent failed to find groups entry for %q", strings.Split(args, " ")[1])
}
return groups[0], nil
}
func callGetent(args string) (io.Reader, error) {
entOnce.Do(func() { getentCmd, _ = resolveBinary("getent") })
// if no `getent` command on host, can't do anything else
if getentCmd == "" {
return nil, fmt.Errorf("")
}
out, err := execCmd(getentCmd, args)
if err != nil {
exitCode, errC := system.GetExitCode(err)
if errC != nil {
return nil, err
}
switch exitCode {
case 1:
return nil, fmt.Errorf("getent reported invalid parameters/database unknown")
case 2:
terms := strings.Split(args, " ")
return nil, fmt.Errorf("getent unable to find entry %q in %s database", terms[1], terms[0])
case 3:
return nil, fmt.Errorf("getent database doesn't support enumeration")
default:
return nil, err
}
}
return bytes.NewReader(out), nil
}
// lazyChown performs a chown only if the uid/gid don't match what's requested
// Normally a Chown is a no-op if uid/gid match, but in some cases this can still cause an error, e.g. if the
// dir is on an NFS share, so don't call chown unless we absolutely must.
func lazyChown(p string, uid, gid int, stat *system.StatT) error {
if stat == nil {
var err error
stat, err = system.Stat(p)
if err != nil {
return err
}
}
if stat.UID() == uint32(uid) && stat.GID() == uint32(gid) {
return nil
}
return os.Chown(p, uid, gid)
}
@@ -0,0 +1,23 @@
package idtools // import "github.com/ory/dockertest/v3/docker/pkg/idtools"
import (
"os"
"github.com/ory/dockertest/v3/docker/pkg/system"
)
// Platforms such as Windows do not support the UID/GID concept. So make this
// just a wrapper around system.MkdirAll.
func mkdirAs(path string, mode os.FileMode, ownerUID, ownerGID int, mkAll, chownExisting bool) error {
if err := system.MkdirAll(path, mode, ""); err != nil {
return err
}
return nil
}
// CanAccess takes a valid (existing) directory and a uid, gid pair and determines
// if that uid, gid pair has access (execute bit) to the directory
// Windows does not require/support this function, so always return true
func CanAccess(path string, pair IDPair) bool {
return true
}
@@ -0,0 +1,164 @@
package idtools // import "github.com/ory/dockertest/v3/docker/pkg/idtools"
import (
"fmt"
"regexp"
"sort"
"strconv"
"strings"
"sync"
)
// add a user and/or group to Linux /etc/passwd, /etc/group using standard
// Linux distribution commands:
// adduser --system --shell /bin/false --disabled-login --disabled-password --no-create-home --group <username>
// useradd -r -s /bin/false <username>
var (
once sync.Once
userCommand string
cmdTemplates = map[string]string{
"adduser": "--system --shell /bin/false --no-create-home --disabled-login --disabled-password --group %s",
"useradd": "-r -s /bin/false %s",
"usermod": "-%s %d-%d %s",
}
idOutRegexp = regexp.MustCompile(`uid=([0-9]+).*gid=([0-9]+)`)
// default length for a UID/GID subordinate range
defaultRangeLen = 65536
defaultRangeStart = 100000
userMod = "usermod"
)
// AddNamespaceRangesUser takes a username and uses the standard system
// utility to create a system user/group pair used to hold the
// /etc/sub{uid,gid} ranges which will be used for user namespace
// mapping ranges in containers.
func AddNamespaceRangesUser(name string) (int, int, error) {
if err := addUser(name); err != nil {
return -1, -1, fmt.Errorf("Error adding user %q: %v", name, err)
}
// Query the system for the created uid and gid pair
out, err := execCmd("id", name)
if err != nil {
return -1, -1, fmt.Errorf("Error trying to find uid/gid for new user %q: %v", name, err)
}
matches := idOutRegexp.FindStringSubmatch(strings.TrimSpace(string(out)))
if len(matches) != 3 {
return -1, -1, fmt.Errorf("Can't find uid, gid from `id` output: %q", string(out))
}
uid, err := strconv.Atoi(matches[1])
if err != nil {
return -1, -1, fmt.Errorf("Can't convert found uid (%s) to int: %v", matches[1], err)
}
gid, err := strconv.Atoi(matches[2])
if err != nil {
return -1, -1, fmt.Errorf("Can't convert found gid (%s) to int: %v", matches[2], err)
}
// Now we need to create the subuid/subgid ranges for our new user/group (system users
// do not get auto-created ranges in subuid/subgid)
if err := createSubordinateRanges(name); err != nil {
return -1, -1, fmt.Errorf("Couldn't create subordinate ID ranges: %v", err)
}
return uid, gid, nil
}
func addUser(userName string) error {
once.Do(func() {
// set up which commands are used for adding users/groups dependent on distro
if _, err := resolveBinary("adduser"); err == nil {
userCommand = "adduser"
} else if _, err := resolveBinary("useradd"); err == nil {
userCommand = "useradd"
}
})
if userCommand == "" {
return fmt.Errorf("Cannot add user; no useradd/adduser binary found")
}
args := fmt.Sprintf(cmdTemplates[userCommand], userName)
out, err := execCmd(userCommand, args)
if err != nil {
return fmt.Errorf("Failed to add user with error: %v; output: %q", err, string(out))
}
return nil
}
func createSubordinateRanges(name string) error {
// first, we should verify that ranges weren't automatically created
// by the distro tooling
ranges, err := parseSubuid(name)
if err != nil {
return fmt.Errorf("Error while looking for subuid ranges for user %q: %v", name, err)
}
if len(ranges) == 0 {
// no UID ranges; let's create one
startID, err := findNextUIDRange()
if err != nil {
return fmt.Errorf("Can't find available subuid range: %v", err)
}
out, err := execCmd(userMod, fmt.Sprintf(cmdTemplates[userMod], "v", startID, startID+defaultRangeLen-1, name))
if err != nil {
return fmt.Errorf("Unable to add subuid range to user: %q; output: %s, err: %v", name, out, err)
}
}
ranges, err = parseSubgid(name)
if err != nil {
return fmt.Errorf("Error while looking for subgid ranges for user %q: %v", name, err)
}
if len(ranges) == 0 {
// no GID ranges; let's create one
startID, err := findNextGIDRange()
if err != nil {
return fmt.Errorf("Can't find available subgid range: %v", err)
}
out, err := execCmd(userMod, fmt.Sprintf(cmdTemplates[userMod], "w", startID, startID+defaultRangeLen-1, name))
if err != nil {
return fmt.Errorf("Unable to add subgid range to user: %q; output: %s, err: %v", name, out, err)
}
}
return nil
}
func findNextUIDRange() (int, error) {
ranges, err := parseSubuid("ALL")
if err != nil {
return -1, fmt.Errorf("Couldn't parse all ranges in /etc/subuid file: %v", err)
}
sort.Sort(ranges)
return findNextRangeStart(ranges)
}
func findNextGIDRange() (int, error) {
ranges, err := parseSubgid("ALL")
if err != nil {
return -1, fmt.Errorf("Couldn't parse all ranges in /etc/subgid file: %v", err)
}
sort.Sort(ranges)
return findNextRangeStart(ranges)
}
func findNextRangeStart(rangeList ranges) (int, error) {
startID := defaultRangeStart
for _, arange := range rangeList {
if wouldOverlap(arange, startID) {
startID = arange.Start + arange.Length
}
}
return startID, nil
}
func wouldOverlap(arange subIDRange, ID int) bool {
low := ID
high := ID + defaultRangeLen
if (low >= arange.Start && low <= arange.Start+arange.Length) ||
(high <= arange.Start+arange.Length && high >= arange.Start) {
return true
}
return false
}
@@ -0,0 +1,12 @@
// +build !linux
package idtools // import "github.com/ory/dockertest/v3/docker/pkg/idtools"
import "fmt"
// AddNamespaceRangesUser takes a name and finds an unused uid, gid pair
// and calls the appropriate helper function to add the group and then
// the user to the group in /etc/group and /etc/passwd respectively.
func AddNamespaceRangesUser(name string) (int, int, error) {
return -1, -1, fmt.Errorf("No support for adding users or groups on this OS")
}
+32
View File
@@ -0,0 +1,32 @@
// +build !windows
package idtools // import "github.com/ory/dockertest/v3/docker/pkg/idtools"
import (
"fmt"
"os/exec"
"path/filepath"
"strings"
)
func resolveBinary(binname string) (string, error) {
binaryPath, err := exec.LookPath(binname)
if err != nil {
return "", err
}
resolvedPath, err := filepath.EvalSymlinks(binaryPath)
if err != nil {
return "", err
}
//only return no error if the final resolved binary basename
//matches what was searched for
if filepath.Base(resolvedPath) == binname {
return resolvedPath, nil
}
return "", fmt.Errorf("Binary %q does not resolve to a binary of that name in $PATH (%q)", binname, resolvedPath)
}
func execCmd(cmd, args string) ([]byte, error) {
execCmd := exec.Command(cmd, strings.Split(args, " ")...)
return execCmd.CombinedOutput()
}
+51
View File
@@ -0,0 +1,51 @@
package ioutils // import "github.com/ory/dockertest/v3/docker/pkg/ioutils"
import (
"errors"
"io"
)
var errBufferFull = errors.New("buffer is full")
type fixedBuffer struct {
buf []byte
pos int
lastRead int
}
func (b *fixedBuffer) Write(p []byte) (int, error) {
n := copy(b.buf[b.pos:cap(b.buf)], p)
b.pos += n
if n < len(p) {
if b.pos == cap(b.buf) {
return n, errBufferFull
}
return n, io.ErrShortWrite
}
return n, nil
}
func (b *fixedBuffer) Read(p []byte) (int, error) {
n := copy(p, b.buf[b.lastRead:b.pos])
b.lastRead += n
return n, nil
}
func (b *fixedBuffer) Len() int {
return b.pos - b.lastRead
}
func (b *fixedBuffer) Cap() int {
return cap(b.buf)
}
func (b *fixedBuffer) Reset() {
b.pos = 0
b.lastRead = 0
b.buf = b.buf[:0]
}
func (b *fixedBuffer) String() string {
return string(b.buf[b.lastRead:b.pos])
}
+186
View File
@@ -0,0 +1,186 @@
package ioutils // import "github.com/ory/dockertest/v3/docker/pkg/ioutils"
import (
"errors"
"io"
"sync"
)
// maxCap is the highest capacity to use in byte slices that buffer data.
const maxCap = 1e6
// minCap is the lowest capacity to use in byte slices that buffer data
const minCap = 64
// blockThreshold is the minimum number of bytes in the buffer which will cause
// a write to BytesPipe to block when allocating a new slice.
const blockThreshold = 1e6
var (
// ErrClosed is returned when Write is called on a closed BytesPipe.
ErrClosed = errors.New("write to closed BytesPipe")
bufPools = make(map[int]*sync.Pool)
bufPoolsLock sync.Mutex
)
// BytesPipe is io.ReadWriteCloser which works similarly to pipe(queue).
// All written data may be read at most once. Also, BytesPipe allocates
// and releases new byte slices to adjust to current needs, so the buffer
// won't be overgrown after peak loads.
type BytesPipe struct {
mu sync.Mutex
wait *sync.Cond
buf []*fixedBuffer
bufLen int
closeErr error // error to return from next Read. set to nil if not closed.
}
// NewBytesPipe creates new BytesPipe, initialized by specified slice.
// If buf is nil, then it will be initialized with slice which cap is 64.
// buf will be adjusted in a way that len(buf) == 0, cap(buf) == cap(buf).
func NewBytesPipe() *BytesPipe {
bp := &BytesPipe{}
bp.buf = append(bp.buf, getBuffer(minCap))
bp.wait = sync.NewCond(&bp.mu)
return bp
}
// Write writes p to BytesPipe.
// It can allocate new []byte slices in a process of writing.
func (bp *BytesPipe) Write(p []byte) (int, error) {
bp.mu.Lock()
written := 0
loop0:
for {
if bp.closeErr != nil {
bp.mu.Unlock()
return written, ErrClosed
}
if len(bp.buf) == 0 {
bp.buf = append(bp.buf, getBuffer(64))
}
// get the last buffer
b := bp.buf[len(bp.buf)-1]
n, err := b.Write(p)
written += n
bp.bufLen += n
// errBufferFull is an error we expect to get if the buffer is full
if err != nil && err != errBufferFull {
bp.wait.Broadcast()
bp.mu.Unlock()
return written, err
}
// if there was enough room to write all then break
if len(p) == n {
break
}
// more data: write to the next slice
p = p[n:]
// make sure the buffer doesn't grow too big from this write
for bp.bufLen >= blockThreshold {
bp.wait.Wait()
if bp.closeErr != nil {
continue loop0
}
}
// add new byte slice to the buffers slice and continue writing
nextCap := b.Cap() * 2
if nextCap > maxCap {
nextCap = maxCap
}
bp.buf = append(bp.buf, getBuffer(nextCap))
}
bp.wait.Broadcast()
bp.mu.Unlock()
return written, nil
}
// CloseWithError causes further reads from a BytesPipe to return immediately.
func (bp *BytesPipe) CloseWithError(err error) error {
bp.mu.Lock()
if err != nil {
bp.closeErr = err
} else {
bp.closeErr = io.EOF
}
bp.wait.Broadcast()
bp.mu.Unlock()
return nil
}
// Close causes further reads from a BytesPipe to return immediately.
func (bp *BytesPipe) Close() error {
return bp.CloseWithError(nil)
}
// Read reads bytes from BytesPipe.
// Data could be read only once.
func (bp *BytesPipe) Read(p []byte) (n int, err error) {
bp.mu.Lock()
if bp.bufLen == 0 {
if bp.closeErr != nil {
bp.mu.Unlock()
return 0, bp.closeErr
}
bp.wait.Wait()
if bp.bufLen == 0 && bp.closeErr != nil {
err := bp.closeErr
bp.mu.Unlock()
return 0, err
}
}
for bp.bufLen > 0 {
b := bp.buf[0]
read, _ := b.Read(p) // ignore error since fixedBuffer doesn't really return an error
n += read
bp.bufLen -= read
if b.Len() == 0 {
// it's empty so return it to the pool and move to the next one
returnBuffer(b)
bp.buf[0] = nil
bp.buf = bp.buf[1:]
}
if len(p) == read {
break
}
p = p[read:]
}
bp.wait.Broadcast()
bp.mu.Unlock()
return
}
func returnBuffer(b *fixedBuffer) {
b.Reset()
bufPoolsLock.Lock()
pool := bufPools[b.Cap()]
bufPoolsLock.Unlock()
if pool != nil {
pool.Put(b)
}
}
func getBuffer(size int) *fixedBuffer {
bufPoolsLock.Lock()
pool, ok := bufPools[size]
if !ok {
pool = &sync.Pool{New: func() interface{} { return &fixedBuffer{buf: make([]byte, 0, size)} }}
bufPools[size] = pool
}
bufPoolsLock.Unlock()
return pool.Get().(*fixedBuffer)
}
+162
View File
@@ -0,0 +1,162 @@
package ioutils // import "github.com/ory/dockertest/v3/docker/pkg/ioutils"
import (
"io"
"io/ioutil"
"os"
"path/filepath"
)
// NewAtomicFileWriter returns WriteCloser so that writing to it writes to a
// temporary file and closing it atomically changes the temporary file to
// destination path. Writing and closing concurrently is not allowed.
func NewAtomicFileWriter(filename string, perm os.FileMode) (io.WriteCloser, error) {
f, err := ioutil.TempFile(filepath.Dir(filename), ".tmp-"+filepath.Base(filename))
if err != nil {
return nil, err
}
abspath, err := filepath.Abs(filename)
if err != nil {
return nil, err
}
return &atomicFileWriter{
f: f,
fn: abspath,
perm: perm,
}, nil
}
// AtomicWriteFile atomically writes data to a file named by filename.
func AtomicWriteFile(filename string, data []byte, perm os.FileMode) error {
f, err := NewAtomicFileWriter(filename, perm)
if err != nil {
return err
}
n, err := f.Write(data)
if err == nil && n < len(data) {
err = io.ErrShortWrite
f.(*atomicFileWriter).writeErr = err
}
if err1 := f.Close(); err == nil {
err = err1
}
return err
}
type atomicFileWriter struct {
f *os.File
fn string
writeErr error
perm os.FileMode
}
func (w *atomicFileWriter) Write(dt []byte) (int, error) {
n, err := w.f.Write(dt)
if err != nil {
w.writeErr = err
}
return n, err
}
func (w *atomicFileWriter) Close() (retErr error) {
defer func() {
if retErr != nil || w.writeErr != nil {
os.Remove(w.f.Name())
}
}()
if err := w.f.Sync(); err != nil {
w.f.Close()
return err
}
if err := w.f.Close(); err != nil {
return err
}
if err := os.Chmod(w.f.Name(), w.perm); err != nil {
return err
}
if w.writeErr == nil {
return os.Rename(w.f.Name(), w.fn)
}
return nil
}
// AtomicWriteSet is used to atomically write a set
// of files and ensure they are visible at the same time.
// Must be committed to a new directory.
type AtomicWriteSet struct {
root string
}
// NewAtomicWriteSet creates a new atomic write set to
// atomically create a set of files. The given directory
// is used as the base directory for storing files before
// commit. If no temporary directory is given the system
// default is used.
func NewAtomicWriteSet(tmpDir string) (*AtomicWriteSet, error) {
td, err := ioutil.TempDir(tmpDir, "write-set-")
if err != nil {
return nil, err
}
return &AtomicWriteSet{
root: td,
}, nil
}
// WriteFile writes a file to the set, guaranteeing the file
// has been synced.
func (ws *AtomicWriteSet) WriteFile(filename string, data []byte, perm os.FileMode) error {
f, err := ws.FileWriter(filename, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, perm)
if err != nil {
return err
}
n, err := f.Write(data)
if err == nil && n < len(data) {
err = io.ErrShortWrite
}
if err1 := f.Close(); err == nil {
err = err1
}
return err
}
type syncFileCloser struct {
*os.File
}
func (w syncFileCloser) Close() error {
err := w.File.Sync()
if err1 := w.File.Close(); err == nil {
err = err1
}
return err
}
// FileWriter opens a file writer inside the set. The file
// should be synced and closed before calling commit.
func (ws *AtomicWriteSet) FileWriter(name string, flag int, perm os.FileMode) (io.WriteCloser, error) {
f, err := os.OpenFile(filepath.Join(ws.root, name), flag, perm)
if err != nil {
return nil, err
}
return syncFileCloser{f}, nil
}
// Cancel cancels the set and removes all temporary data
// created in the set.
func (ws *AtomicWriteSet) Cancel() error {
return os.RemoveAll(ws.root)
}
// Commit moves all created files to the target directory. The
// target directory must not exist and the parent of the target
// directory must exist.
func (ws *AtomicWriteSet) Commit(target string) error {
return os.Rename(ws.root, target)
}
// String returns the location the set is writing to.
func (ws *AtomicWriteSet) String() string {
return ws.root
}
+158
View File
@@ -0,0 +1,158 @@
package ioutils // import "github.com/ory/dockertest/v3/docker/pkg/ioutils"
import (
"crypto/sha256"
"encoding/hex"
"io"
"golang.org/x/net/context"
)
// ReadCloserWrapper wraps an io.Reader, and implements an io.ReadCloser
// It calls the given callback function when closed. It should be constructed
// with NewReadCloserWrapper
type ReadCloserWrapper struct {
io.Reader
closer func() error
}
// Close calls back the passed closer function
func (r *ReadCloserWrapper) Close() error {
return r.closer()
}
// NewReadCloserWrapper returns a new io.ReadCloser.
func NewReadCloserWrapper(r io.Reader, closer func() error) io.ReadCloser {
return &ReadCloserWrapper{
Reader: r,
closer: closer,
}
}
type readerErrWrapper struct {
reader io.Reader
closer func()
}
func (r *readerErrWrapper) Read(p []byte) (int, error) {
n, err := r.reader.Read(p)
if err != nil {
r.closer()
}
return n, err
}
// NewReaderErrWrapper returns a new io.Reader.
func NewReaderErrWrapper(r io.Reader, closer func()) io.Reader {
return &readerErrWrapper{
reader: r,
closer: closer,
}
}
// HashData returns the sha256 sum of src.
func HashData(src io.Reader) (string, error) {
h := sha256.New()
if _, err := io.Copy(h, src); err != nil {
return "", err
}
return "sha256:" + hex.EncodeToString(h.Sum(nil)), nil
}
// OnEOFReader wraps an io.ReadCloser and a function
// the function will run at the end of file or close the file.
type OnEOFReader struct {
Rc io.ReadCloser
Fn func()
}
func (r *OnEOFReader) Read(p []byte) (n int, err error) {
n, err = r.Rc.Read(p)
if err == io.EOF {
r.runFunc()
}
return
}
// Close closes the file and run the function.
func (r *OnEOFReader) Close() error {
err := r.Rc.Close()
r.runFunc()
return err
}
func (r *OnEOFReader) runFunc() {
if fn := r.Fn; fn != nil {
fn()
r.Fn = nil
}
}
// cancelReadCloser wraps an io.ReadCloser with a context for cancelling read
// operations.
type cancelReadCloser struct {
cancel func()
pR *io.PipeReader // Stream to read from
pW *io.PipeWriter
}
// NewCancelReadCloser creates a wrapper that closes the ReadCloser when the
// context is cancelled. The returned io.ReadCloser must be closed when it is
// no longer needed.
func NewCancelReadCloser(ctx context.Context, in io.ReadCloser) io.ReadCloser {
pR, pW := io.Pipe()
// Create a context used to signal when the pipe is closed
doneCtx, cancel := context.WithCancel(context.Background())
p := &cancelReadCloser{
cancel: cancel,
pR: pR,
pW: pW,
}
go func() {
_, err := io.Copy(pW, in)
select {
case <-ctx.Done():
// If the context was closed, p.closeWithError
// was already called. Calling it again would
// change the error that Read returns.
default:
p.closeWithError(err)
}
in.Close()
}()
go func() {
for {
select {
case <-ctx.Done():
p.closeWithError(ctx.Err())
case <-doneCtx.Done():
return
}
}
}()
return p
}
// Read wraps the Read method of the pipe that provides data from the wrapped
// ReadCloser.
func (p *cancelReadCloser) Read(buf []byte) (n int, err error) {
return p.pR.Read(buf)
}
// closeWithError closes the wrapper and its underlying reader. It will
// cause future calls to Read to return err.
func (p *cancelReadCloser) closeWithError(err error) {
p.pW.CloseWithError(err)
p.cancel()
}
// Close closes the wrapper its underlying reader. It will cause
// future calls to Read to return io.EOF.
func (p *cancelReadCloser) Close() error {
p.closeWithError(io.EOF)
return nil
}
+10
View File
@@ -0,0 +1,10 @@
// +build !windows
package ioutils // import "github.com/ory/dockertest/v3/docker/pkg/ioutils"
import "io/ioutil"
// TempDir on Unix systems is equivalent to ioutil.TempDir.
func TempDir(dir, prefix string) (string, error) {
return ioutil.TempDir(dir, prefix)
}
+16
View File
@@ -0,0 +1,16 @@
package ioutils // import "github.com/ory/dockertest/v3/docker/pkg/ioutils"
import (
"io/ioutil"
"github.com/ory/dockertest/v3/docker/pkg/longpath"
)
// TempDir is the equivalent of ioutil.TempDir, except that the result is in Windows longpath format.
func TempDir(dir, prefix string) (string, error) {
tempDir, err := ioutil.TempDir(dir, prefix)
if err != nil {
return "", err
}
return longpath.AddPrefix(tempDir), nil
}
+92
View File
@@ -0,0 +1,92 @@
package ioutils // import "github.com/ory/dockertest/v3/docker/pkg/ioutils"
import (
"io"
"sync"
)
// WriteFlusher wraps the Write and Flush operation ensuring that every write
// is a flush. In addition, the Close method can be called to intercept
// Read/Write calls if the targets lifecycle has already ended.
type WriteFlusher struct {
w io.Writer
flusher flusher
flushed chan struct{}
flushedOnce sync.Once
closed chan struct{}
closeLock sync.Mutex
}
type flusher interface {
Flush()
}
var errWriteFlusherClosed = io.EOF
func (wf *WriteFlusher) Write(b []byte) (n int, err error) {
select {
case <-wf.closed:
return 0, errWriteFlusherClosed
default:
}
n, err = wf.w.Write(b)
wf.Flush() // every write is a flush.
return n, err
}
// Flush the stream immediately.
func (wf *WriteFlusher) Flush() {
select {
case <-wf.closed:
return
default:
}
wf.flushedOnce.Do(func() {
close(wf.flushed)
})
wf.flusher.Flush()
}
// Flushed returns the state of flushed.
// If it's flushed, return true, or else it return false.
func (wf *WriteFlusher) Flushed() bool {
// BUG(stevvooe): Remove this method. Its use is inherently racy. Seems to
// be used to detect whether or a response code has been issued or not.
// Another hook should be used instead.
var flushed bool
select {
case <-wf.flushed:
flushed = true
default:
}
return flushed
}
// Close closes the write flusher, disallowing any further writes to the
// target. After the flusher is closed, all calls to write or flush will
// result in an error.
func (wf *WriteFlusher) Close() error {
wf.closeLock.Lock()
defer wf.closeLock.Unlock()
select {
case <-wf.closed:
return errWriteFlusherClosed
default:
close(wf.closed)
}
return nil
}
// NewWriteFlusher returns a new WriteFlusher.
func NewWriteFlusher(w io.Writer) *WriteFlusher {
var fl flusher
if f, ok := w.(flusher); ok {
fl = f
} else {
fl = &NopFlusher{}
}
return &WriteFlusher{w: w, flusher: fl, closed: make(chan struct{}), flushed: make(chan struct{})}
}
+66
View File
@@ -0,0 +1,66 @@
package ioutils // import "github.com/ory/dockertest/v3/docker/pkg/ioutils"
import "io"
// NopWriter represents a type which write operation is nop.
type NopWriter struct{}
func (*NopWriter) Write(buf []byte) (int, error) {
return len(buf), nil
}
type nopWriteCloser struct {
io.Writer
}
func (w *nopWriteCloser) Close() error { return nil }
// NopWriteCloser returns a nopWriteCloser.
func NopWriteCloser(w io.Writer) io.WriteCloser {
return &nopWriteCloser{w}
}
// NopFlusher represents a type which flush operation is nop.
type NopFlusher struct{}
// Flush is a nop operation.
func (f *NopFlusher) Flush() {}
type writeCloserWrapper struct {
io.Writer
closer func() error
}
func (r *writeCloserWrapper) Close() error {
return r.closer()
}
// NewWriteCloserWrapper returns a new io.WriteCloser.
func NewWriteCloserWrapper(r io.Writer, closer func() error) io.WriteCloser {
return &writeCloserWrapper{
Writer: r,
closer: closer,
}
}
// WriteCounter wraps a concrete io.Writer and hold a count of the number
// of bytes written to the writer during a "session".
// This can be convenient when write return is masked
// (e.g., json.Encoder.Encode())
type WriteCounter struct {
Count int64
Writer io.Writer
}
// NewWriteCounter returns a new WriteCounter.
func NewWriteCounter(w io.Writer) *WriteCounter {
return &WriteCounter{
Writer: w,
}
}
func (wc *WriteCounter) Write(p []byte) (count int, err error) {
count, err = wc.Writer.Write(p)
wc.Count += int64(count)
return
}
@@ -0,0 +1,335 @@
package jsonmessage // import "github.com/ory/dockertest/v3/docker/pkg/jsonmessage"
import (
"encoding/json"
"fmt"
"io"
"os"
"strings"
"time"
gotty "github.com/Nvveen/Gotty"
"github.com/ory/dockertest/v3/docker/pkg/term"
units "github.com/docker/go-units"
)
// RFC3339NanoFixed is time.RFC3339Nano with nanoseconds padded using zeros to
// ensure the formatted time isalways the same number of characters.
const RFC3339NanoFixed = "2006-01-02T15:04:05.000000000Z07:00"
// JSONError wraps a concrete Code and Message, `Code` is
// is an integer error code, `Message` is the error message.
type JSONError struct {
Code int `json:"code,omitempty"`
Message string `json:"message,omitempty"`
}
func (e *JSONError) Error() string {
return e.Message
}
// JSONProgress describes a Progress. terminalFd is the fd of the current terminal,
// Start is the initial value for the operation. Current is the current status and
// value of the progress made towards Total. Total is the end value describing when
// we made 100% progress for an operation.
type JSONProgress struct {
terminalFd uintptr
Current int64 `json:"current,omitempty"`
Total int64 `json:"total,omitempty"`
Start int64 `json:"start,omitempty"`
// If true, don't show xB/yB
HideCounts bool `json:"hidecounts,omitempty"`
Units string `json:"units,omitempty"`
nowFunc func() time.Time
winSize int
}
func (p *JSONProgress) String() string {
var (
width = p.width()
pbBox string
numbersBox string
timeLeftBox string
)
if p.Current <= 0 && p.Total <= 0 {
return ""
}
if p.Total <= 0 {
switch p.Units {
case "":
current := units.HumanSize(float64(p.Current))
return fmt.Sprintf("%8v", current)
default:
return fmt.Sprintf("%d %s", p.Current, p.Units)
}
}
percentage := int(float64(p.Current)/float64(p.Total)*100) / 2
if percentage > 50 {
percentage = 50
}
if width > 110 {
// this number can't be negative gh#7136
numSpaces := 0
if 50-percentage > 0 {
numSpaces = 50 - percentage
}
pbBox = fmt.Sprintf("[%s>%s] ", strings.Repeat("=", percentage), strings.Repeat(" ", numSpaces))
}
switch {
case p.HideCounts:
case p.Units == "": // no units, use bytes
current := units.HumanSize(float64(p.Current))
total := units.HumanSize(float64(p.Total))
numbersBox = fmt.Sprintf("%8v/%v", current, total)
if p.Current > p.Total {
// remove total display if the reported current is wonky.
numbersBox = fmt.Sprintf("%8v", current)
}
default:
numbersBox = fmt.Sprintf("%d/%d %s", p.Current, p.Total, p.Units)
if p.Current > p.Total {
// remove total display if the reported current is wonky.
numbersBox = fmt.Sprintf("%d %s", p.Current, p.Units)
}
}
if p.Current > 0 && p.Start > 0 && percentage < 50 {
fromStart := p.now().Sub(time.Unix(p.Start, 0))
perEntry := fromStart / time.Duration(p.Current)
left := time.Duration(p.Total-p.Current) * perEntry
left = (left / time.Second) * time.Second
if width > 50 {
timeLeftBox = " " + left.String()
}
}
return pbBox + numbersBox + timeLeftBox
}
// shim for testing
func (p *JSONProgress) now() time.Time {
if p.nowFunc == nil {
p.nowFunc = func() time.Time {
return time.Now().UTC()
}
}
return p.nowFunc()
}
// shim for testing
func (p *JSONProgress) width() int {
if p.winSize != 0 {
return p.winSize
}
ws, err := term.GetWinsize(p.terminalFd)
if err == nil {
return int(ws.Width)
}
return 200
}
// JSONMessage defines a message struct. It describes
// the created time, where it from, status, ID of the
// message. It's used for docker events.
type JSONMessage struct {
Stream string `json:"stream,omitempty"`
Status string `json:"status,omitempty"`
Progress *JSONProgress `json:"progressDetail,omitempty"`
ProgressMessage string `json:"progress,omitempty"` //deprecated
ID string `json:"id,omitempty"`
From string `json:"from,omitempty"`
Time int64 `json:"time,omitempty"`
TimeNano int64 `json:"timeNano,omitempty"`
Error *JSONError `json:"errorDetail,omitempty"`
ErrorMessage string `json:"error,omitempty"` //deprecated
// Aux contains out-of-band data, such as digests for push signing and image id after building.
Aux *json.RawMessage `json:"aux,omitempty"`
}
/* Satisfied by gotty.TermInfo as well as noTermInfo from below */
type termInfo interface {
Parse(attr string, params ...interface{}) (string, error)
}
type noTermInfo struct{} // canary used when no terminfo.
func (ti *noTermInfo) Parse(attr string, params ...interface{}) (string, error) {
return "", fmt.Errorf("noTermInfo")
}
func clearLine(out io.Writer, ti termInfo) {
// el2 (clear whole line) is not exposed by terminfo.
// First clear line from beginning to cursor
if attr, err := ti.Parse("el1"); err == nil {
fmt.Fprintf(out, "%s", attr)
} else {
fmt.Fprintf(out, "\x1b[1K")
}
// Then clear line from cursor to end
if attr, err := ti.Parse("el"); err == nil {
fmt.Fprintf(out, "%s", attr)
} else {
fmt.Fprintf(out, "\x1b[K")
}
}
func cursorUp(out io.Writer, ti termInfo, l int) {
if l == 0 { // Should never be the case, but be tolerant
return
}
if attr, err := ti.Parse("cuu", l); err == nil {
fmt.Fprintf(out, "%s", attr)
} else {
fmt.Fprintf(out, "\x1b[%dA", l)
}
}
func cursorDown(out io.Writer, ti termInfo, l int) {
if l == 0 { // Should never be the case, but be tolerant
return
}
if attr, err := ti.Parse("cud", l); err == nil {
fmt.Fprintf(out, "%s", attr)
} else {
fmt.Fprintf(out, "\x1b[%dB", l)
}
}
// Display displays the JSONMessage to `out`. `termInfo` is non-nil if `out`
// is a terminal. If this is the case, it will erase the entire current line
// when displaying the progressbar.
func (jm *JSONMessage) Display(out io.Writer, termInfo termInfo) error {
if jm.Error != nil {
if jm.Error.Code == 401 {
return fmt.Errorf("authentication is required")
}
return jm.Error
}
var endl string
if termInfo != nil && jm.Stream == "" && jm.Progress != nil {
clearLine(out, termInfo)
endl = "\r"
fmt.Fprintf(out, endl)
} else if jm.Progress != nil && jm.Progress.String() != "" { //disable progressbar in non-terminal
return nil
}
if jm.TimeNano != 0 {
fmt.Fprintf(out, "%s ", time.Unix(0, jm.TimeNano).Format(RFC3339NanoFixed))
} else if jm.Time != 0 {
fmt.Fprintf(out, "%s ", time.Unix(jm.Time, 0).Format(RFC3339NanoFixed))
}
if jm.ID != "" {
fmt.Fprintf(out, "%s: ", jm.ID)
}
if jm.From != "" {
fmt.Fprintf(out, "(from %s) ", jm.From)
}
if jm.Progress != nil && termInfo != nil {
fmt.Fprintf(out, "%s %s%s", jm.Status, jm.Progress.String(), endl)
} else if jm.ProgressMessage != "" { //deprecated
fmt.Fprintf(out, "%s %s%s", jm.Status, jm.ProgressMessage, endl)
} else if jm.Stream != "" {
fmt.Fprintf(out, "%s%s", jm.Stream, endl)
} else {
fmt.Fprintf(out, "%s%s\n", jm.Status, endl)
}
return nil
}
// DisplayJSONMessagesStream displays a json message stream from `in` to `out`, `isTerminal`
// describes if `out` is a terminal. If this is the case, it will print `\n` at the end of
// each line and move the cursor while displaying.
func DisplayJSONMessagesStream(in io.Reader, out io.Writer, terminalFd uintptr, isTerminal bool, auxCallback func(*json.RawMessage)) error {
var (
dec = json.NewDecoder(in)
ids = make(map[string]int)
)
var termInfo termInfo
if isTerminal {
term := os.Getenv("TERM")
if term == "" {
term = "vt102"
}
var err error
if termInfo, err = gotty.OpenTermInfo(term); err != nil {
termInfo = &noTermInfo{}
}
}
for {
diff := 0
var jm JSONMessage
if err := dec.Decode(&jm); err != nil {
if err == io.EOF {
break
}
return err
}
if jm.Aux != nil {
if auxCallback != nil {
auxCallback(jm.Aux)
}
continue
}
if jm.Progress != nil {
jm.Progress.terminalFd = terminalFd
}
if jm.ID != "" && (jm.Progress != nil || jm.ProgressMessage != "") {
line, ok := ids[jm.ID]
if !ok {
// NOTE: This approach of using len(id) to
// figure out the number of lines of history
// only works as long as we clear the history
// when we output something that's not
// accounted for in the map, such as a line
// with no ID.
line = len(ids)
ids[jm.ID] = line
if termInfo != nil {
fmt.Fprintf(out, "\n")
}
}
diff = len(ids) - line
if termInfo != nil {
cursorUp(out, termInfo, diff)
}
} else {
// When outputting something that isn't progress
// output, clear the history of previous lines. We
// don't want progress entries from some previous
// operation to be updated (for example, pull -a
// with multiple tags).
ids = make(map[string]int)
}
err := jm.Display(out, termInfo)
if jm.ID != "" && termInfo != nil {
cursorDown(out, termInfo, diff)
}
if err != nil {
return err
}
}
return nil
}
type stream interface {
io.Writer
FD() uintptr
IsTerminal() bool
}
// DisplayJSONMessagesToStream prints json messages to the output stream
func DisplayJSONMessagesToStream(in io.Reader, stream stream, auxCallback func(*json.RawMessage)) error {
return DisplayJSONMessagesStream(in, stream, stream.FD(), stream.IsTerminal(), auxCallback)
}
+26
View File
@@ -0,0 +1,26 @@
// longpath introduces some constants and helper functions for handling long paths
// in Windows, which are expected to be prepended with `\\?\` and followed by either
// a drive letter, a UNC server\share, or a volume identifier.
package longpath // import "github.com/ory/dockertest/v3/docker/pkg/longpath"
import (
"strings"
)
// Prefix is the longpath prefix for Windows file paths.
const Prefix = `\\?\`
// AddPrefix will add the Windows long path prefix to the path provided if
// it does not already have it.
func AddPrefix(path string) string {
if !strings.HasPrefix(path, Prefix) {
if strings.HasPrefix(path, `\\`) {
// This is a UNC path, so we need to add 'UNC' to the path as well.
path = Prefix + `UNC` + path[1:]
} else {
path = Prefix + path
}
}
return path
}
+149
View File
@@ -0,0 +1,149 @@
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
import (
"fmt"
"strings"
)
var flags = map[string]struct {
clear bool
flag int
}{
"defaults": {false, 0},
"ro": {false, RDONLY},
"rw": {true, RDONLY},
"suid": {true, NOSUID},
"nosuid": {false, NOSUID},
"dev": {true, NODEV},
"nodev": {false, NODEV},
"exec": {true, NOEXEC},
"noexec": {false, NOEXEC},
"sync": {false, SYNCHRONOUS},
"async": {true, SYNCHRONOUS},
"dirsync": {false, DIRSYNC},
"remount": {false, REMOUNT},
"mand": {false, MANDLOCK},
"nomand": {true, MANDLOCK},
"atime": {true, NOATIME},
"noatime": {false, NOATIME},
"diratime": {true, NODIRATIME},
"nodiratime": {false, NODIRATIME},
"bind": {false, BIND},
"rbind": {false, RBIND},
"unbindable": {false, UNBINDABLE},
"runbindable": {false, RUNBINDABLE},
"private": {false, PRIVATE},
"rprivate": {false, RPRIVATE},
"shared": {false, SHARED},
"rshared": {false, RSHARED},
"slave": {false, SLAVE},
"rslave": {false, RSLAVE},
"relatime": {false, RELATIME},
"norelatime": {true, RELATIME},
"strictatime": {false, STRICTATIME},
"nostrictatime": {true, STRICTATIME},
}
var validFlags = map[string]bool{
"": true,
"size": true,
"mode": true,
"uid": true,
"gid": true,
"nr_inodes": true,
"nr_blocks": true,
"mpol": true,
}
var propagationFlags = map[string]bool{
"bind": true,
"rbind": true,
"unbindable": true,
"runbindable": true,
"private": true,
"rprivate": true,
"shared": true,
"rshared": true,
"slave": true,
"rslave": true,
}
// MergeTmpfsOptions merge mount options to make sure there is no duplicate.
func MergeTmpfsOptions(options []string) ([]string, error) {
// We use collisions maps to remove duplicates.
// For flag, the key is the flag value (the key for propagation flag is -1)
// For data=value, the key is the data
flagCollisions := map[int]bool{}
dataCollisions := map[string]bool{}
var newOptions []string
// We process in reverse order
for i := len(options) - 1; i >= 0; i-- {
option := options[i]
if option == "defaults" {
continue
}
if f, ok := flags[option]; ok && f.flag != 0 {
// There is only one propagation mode
key := f.flag
if propagationFlags[option] {
key = -1
}
// Check to see if there is collision for flag
if !flagCollisions[key] {
// We prepend the option and add to collision map
newOptions = append([]string{option}, newOptions...)
flagCollisions[key] = true
}
continue
}
opt := strings.SplitN(option, "=", 2)
if len(opt) != 2 || !validFlags[opt[0]] {
return nil, fmt.Errorf("Invalid tmpfs option %q", opt)
}
if !dataCollisions[opt[0]] {
// We prepend the option and add to collision map
newOptions = append([]string{option}, newOptions...)
dataCollisions[opt[0]] = true
}
}
return newOptions, nil
}
// Parse fstab type mount options into mount() flags
// and device specific data
func parseOptions(options string) (int, string) {
var (
flag int
data []string
)
for _, o := range strings.Split(options, ",") {
// If the option does not exist in the flags table or the flag
// is not supported on the platform,
// then it is a data value for a specific fs type
if f, exists := flags[o]; exists && f.flag != 0 {
if f.clear {
flag &= ^f.flag
} else {
flag |= f.flag
}
} else {
data = append(data, o)
}
}
return flag, strings.Join(data, ",")
}
// ParseTmpfsOptions parse fstab type mount options into flags and data
func ParseTmpfsOptions(options string) (int, string, error) {
flags, data := parseOptions(options)
for _, o := range strings.Split(data, ",") {
opt := strings.SplitN(o, "=", 2)
if !validFlags[opt[0]] {
return 0, "", fmt.Errorf("Invalid tmpfs option %q", opt)
}
}
return flags, data, nil
}
+49
View File
@@ -0,0 +1,49 @@
// +build freebsd,cgo
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
/*
#include <sys/mount.h>
*/
import "C"
const (
// RDONLY will mount the filesystem as read-only.
RDONLY = C.MNT_RDONLY
// NOSUID will not allow set-user-identifier or set-group-identifier bits to
// take effect.
NOSUID = C.MNT_NOSUID
// NOEXEC will not allow execution of any binaries on the mounted file system.
NOEXEC = C.MNT_NOEXEC
// SYNCHRONOUS will allow any I/O to the file system to be done synchronously.
SYNCHRONOUS = C.MNT_SYNCHRONOUS
// NOATIME will not update the file access time when reading from a file.
NOATIME = C.MNT_NOATIME
)
// These flags are unsupported.
const (
BIND = 0
DIRSYNC = 0
MANDLOCK = 0
NODEV = 0
NODIRATIME = 0
UNBINDABLE = 0
RUNBINDABLE = 0
PRIVATE = 0
RPRIVATE = 0
SHARED = 0
RSHARED = 0
SLAVE = 0
RSLAVE = 0
RBIND = 0
RELATIVE = 0
RELATIME = 0
REMOUNT = 0
STRICTATIME = 0
mntDetach = 0
)
+87
View File
@@ -0,0 +1,87 @@
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
import (
"golang.org/x/sys/unix"
)
const (
// RDONLY will mount the file system read-only.
RDONLY = unix.MS_RDONLY
// NOSUID will not allow set-user-identifier or set-group-identifier bits to
// take effect.
NOSUID = unix.MS_NOSUID
// NODEV will not interpret character or block special devices on the file
// system.
NODEV = unix.MS_NODEV
// NOEXEC will not allow execution of any binaries on the mounted file system.
NOEXEC = unix.MS_NOEXEC
// SYNCHRONOUS will allow I/O to the file system to be done synchronously.
SYNCHRONOUS = unix.MS_SYNCHRONOUS
// DIRSYNC will force all directory updates within the file system to be done
// synchronously. This affects the following system calls: create, link,
// unlink, symlink, mkdir, rmdir, mknod and rename.
DIRSYNC = unix.MS_DIRSYNC
// REMOUNT will attempt to remount an already-mounted file system. This is
// commonly used to change the mount flags for a file system, especially to
// make a readonly file system writeable. It does not change device or mount
// point.
REMOUNT = unix.MS_REMOUNT
// MANDLOCK will force mandatory locks on a filesystem.
MANDLOCK = unix.MS_MANDLOCK
// NOATIME will not update the file access time when reading from a file.
NOATIME = unix.MS_NOATIME
// NODIRATIME will not update the directory access time.
NODIRATIME = unix.MS_NODIRATIME
// BIND remounts a subtree somewhere else.
BIND = unix.MS_BIND
// RBIND remounts a subtree and all possible submounts somewhere else.
RBIND = unix.MS_BIND | unix.MS_REC
// UNBINDABLE creates a mount which cannot be cloned through a bind operation.
UNBINDABLE = unix.MS_UNBINDABLE
// RUNBINDABLE marks the entire mount tree as UNBINDABLE.
RUNBINDABLE = unix.MS_UNBINDABLE | unix.MS_REC
// PRIVATE creates a mount which carries no propagation abilities.
PRIVATE = unix.MS_PRIVATE
// RPRIVATE marks the entire mount tree as PRIVATE.
RPRIVATE = unix.MS_PRIVATE | unix.MS_REC
// SLAVE creates a mount which receives propagation from its master, but not
// vice versa.
SLAVE = unix.MS_SLAVE
// RSLAVE marks the entire mount tree as SLAVE.
RSLAVE = unix.MS_SLAVE | unix.MS_REC
// SHARED creates a mount which provides the ability to create mirrors of
// that mount such that mounts and unmounts within any of the mirrors
// propagate to the other mirrors.
SHARED = unix.MS_SHARED
// RSHARED marks the entire mount tree as SHARED.
RSHARED = unix.MS_SHARED | unix.MS_REC
// RELATIME updates inode access times relative to modify or change time.
RELATIME = unix.MS_RELATIME
// STRICTATIME allows to explicitly request full atime updates. This makes
// it possible for the kernel to default to relatime or noatime but still
// allow userspace to override it.
STRICTATIME = unix.MS_STRICTATIME
mntDetach = unix.MNT_DETACH
)
@@ -0,0 +1,31 @@
// +build !linux,!freebsd freebsd,!cgo
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
// These flags are unsupported.
const (
BIND = 0
DIRSYNC = 0
MANDLOCK = 0
NOATIME = 0
NODEV = 0
NODIRATIME = 0
NOEXEC = 0
NOSUID = 0
UNBINDABLE = 0
RUNBINDABLE = 0
PRIVATE = 0
RPRIVATE = 0
SHARED = 0
RSHARED = 0
SLAVE = 0
RSLAVE = 0
RBIND = 0
RELATIME = 0
RELATIVE = 0
REMOUNT = 0
STRICTATIME = 0
SYNCHRONOUS = 0
RDONLY = 0
mntDetach = 0
)
+110
View File
@@ -0,0 +1,110 @@
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
import (
"sort"
"strings"
"syscall"
"github.com/sirupsen/logrus"
)
// GetMounts retrieves a list of mounts for the current running process.
func GetMounts() ([]*Info, error) {
return parseMountTable()
}
// Mounted determines if a specified mountpoint has been mounted.
// On Linux it looks at /proc/self/mountinfo.
func Mounted(mountpoint string) (bool, error) {
entries, err := parseMountTable()
if err != nil {
return false, err
}
// Search the table for the mountpoint
for _, e := range entries {
if e.Mountpoint == mountpoint {
return true, nil
}
}
return false, nil
}
// Mount will mount filesystem according to the specified configuration, on the
// condition that the target path is *not* already mounted. Options must be
// specified like the mount or fstab unix commands: "opt1=val1,opt2=val2". See
// flags.go for supported option flags.
func Mount(device, target, mType, options string) error {
flag, _ := parseOptions(options)
if flag&REMOUNT != REMOUNT {
if mounted, err := Mounted(target); err != nil || mounted {
return err
}
}
return ForceMount(device, target, mType, options)
}
// ForceMount will mount a filesystem according to the specified configuration,
// *regardless* if the target path is not already mounted. Options must be
// specified like the mount or fstab unix commands: "opt1=val1,opt2=val2". See
// flags.go for supported option flags.
func ForceMount(device, target, mType, options string) error {
flag, data := parseOptions(options)
return mount(device, target, mType, uintptr(flag), data)
}
// Unmount lazily unmounts a filesystem on supported platforms, otherwise
// does a normal unmount.
func Unmount(target string) error {
if mounted, err := Mounted(target); err != nil || !mounted {
return err
}
return unmount(target, mntDetach)
}
// RecursiveUnmount unmounts the target and all mounts underneath, starting with
// the deepsest mount first.
func RecursiveUnmount(target string) error {
mounts, err := GetMounts()
if err != nil {
return err
}
// Make the deepest mount be first
sort.Slice(mounts, func(i, j int) bool {
return len(mounts[i].Mountpoint) > len(mounts[j].Mountpoint)
})
for i, m := range mounts {
if !strings.HasPrefix(m.Mountpoint, target) {
continue
}
logrus.Debugf("Trying to unmount %s", m.Mountpoint)
err = unmount(m.Mountpoint, mntDetach)
if err != nil {
// If the error is EINVAL either this whole package is wrong (invalid flags passed to unmount(2)) or this is
// not a mountpoint (which is ok in this case).
// Meanwhile calling `Mounted()` is very expensive.
//
// We've purposefully used `syscall.EINVAL` here instead of `unix.EINVAL` to avoid platform branching
// Since `EINVAL` is defined for both Windows and Linux in the `syscall` package (and other platforms),
// this is nicer than defining a custom value that we can refer to in each platform file.
if err == syscall.EINVAL {
continue
}
if i == len(mounts)-1 {
if mounted, e := Mounted(m.Mountpoint); e != nil || mounted {
return err
}
continue
}
// This is some submount, we can ignore this error for now, the final unmount will fail if this is a real problem
logrus.WithError(err).Warnf("Failed to unmount submount %s", m.Mountpoint)
continue
}
logrus.Debugf("Unmounted %s", m.Mountpoint)
}
return nil
}
@@ -0,0 +1,60 @@
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
/*
#include <errno.h>
#include <stdlib.h>
#include <string.h>
#include <sys/_iovec.h>
#include <sys/mount.h>
#include <sys/param.h>
*/
import "C"
import (
"fmt"
"strings"
"unsafe"
"golang.org/x/sys/unix"
)
func allocateIOVecs(options []string) []C.struct_iovec {
out := make([]C.struct_iovec, len(options))
for i, option := range options {
out[i].iov_base = unsafe.Pointer(C.CString(option))
out[i].iov_len = C.size_t(len(option) + 1)
}
return out
}
func mount(device, target, mType string, flag uintptr, data string) error {
isNullFS := false
xs := strings.Split(data, ",")
for _, x := range xs {
if x == "bind" {
isNullFS = true
}
}
options := []string{"fspath", target}
if isNullFS {
options = append(options, "fstype", "nullfs", "target", device)
} else {
options = append(options, "fstype", mType, "from", device)
}
rawOptions := allocateIOVecs(options)
for _, rawOption := range rawOptions {
defer C.free(rawOption.iov_base)
}
if errno := C.nmount(&rawOptions[0], C.uint(len(options)), C.int(flag)); errno != 0 {
reason := C.GoString(C.strerror(*C.__error()))
return fmt.Errorf("Failed to call nmount: %s", reason)
}
return nil
}
func unmount(target string, flag int) error {
return unix.Unmount(target, flag)
}
+57
View File
@@ -0,0 +1,57 @@
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
import (
"golang.org/x/sys/unix"
)
const (
// ptypes is the set propagation types.
ptypes = unix.MS_SHARED | unix.MS_PRIVATE | unix.MS_SLAVE | unix.MS_UNBINDABLE
// pflags is the full set valid flags for a change propagation call.
pflags = ptypes | unix.MS_REC | unix.MS_SILENT
// broflags is the combination of bind and read only
broflags = unix.MS_BIND | unix.MS_RDONLY
)
// isremount returns true if either device name or flags identify a remount request, false otherwise.
func isremount(device string, flags uintptr) bool {
switch {
// We treat device "" and "none" as a remount request to provide compatibility with
// requests that don't explicitly set MS_REMOUNT such as those manipulating bind mounts.
case flags&unix.MS_REMOUNT != 0, device == "", device == "none":
return true
default:
return false
}
}
func mount(device, target, mType string, flags uintptr, data string) error {
oflags := flags &^ ptypes
if !isremount(device, flags) || data != "" {
// Initial call applying all non-propagation flags for mount
// or remount with changed data
if err := unix.Mount(device, target, mType, oflags, data); err != nil {
return err
}
}
if flags&ptypes != 0 {
// Change the propagation type.
if err := unix.Mount("", target, "", flags&pflags, ""); err != nil {
return err
}
}
if oflags&broflags == broflags {
// Remount the bind to apply read only.
return unix.Mount("", target, "", oflags|unix.MS_REMOUNT, "")
}
return nil
}
func unmount(target string, flag int) error {
return unix.Unmount(target, flag)
}
@@ -0,0 +1,11 @@
// +build !linux,!freebsd freebsd,!cgo
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
func mount(device, target, mType string, flag uintptr, data string) error {
panic("Not implemented")
}
func unmount(target string, flag int) error {
panic("Not implemented")
}
+40
View File
@@ -0,0 +1,40 @@
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
// Info reveals information about a particular mounted filesystem. This
// struct is populated from the content in the /proc/<pid>/mountinfo file.
type Info struct {
// ID is a unique identifier of the mount (may be reused after umount).
ID int
// Parent indicates the ID of the mount parent (or of self for the top of the
// mount tree).
Parent int
// Major indicates one half of the device ID which identifies the device class.
Major int
// Minor indicates one half of the device ID which identifies a specific
// instance of device.
Minor int
// Root of the mount within the filesystem.
Root string
// Mountpoint indicates the mount point relative to the process's root.
Mountpoint string
// Opts represents mount-specific options.
Opts string
// Optional represents optional fields.
Optional string
// Fstype indicates the type of filesystem, such as EXT3.
Fstype string
// Source indicates filesystem specific information or "none".
Source string
// VfsOpts represents per super block options.
VfsOpts string
}
@@ -0,0 +1,41 @@
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
/*
#include <sys/param.h>
#include <sys/ucred.h>
#include <sys/mount.h>
*/
import "C"
import (
"fmt"
"reflect"
"unsafe"
)
// Parse /proc/self/mountinfo because comparing Dev and ino does not work from
// bind mounts.
func parseMountTable() ([]*Info, error) {
var rawEntries *C.struct_statfs
count := int(C.getmntinfo(&rawEntries, C.MNT_WAIT))
if count == 0 {
return nil, fmt.Errorf("Failed to call getmntinfo")
}
var entries []C.struct_statfs
header := (*reflect.SliceHeader)(unsafe.Pointer(&entries))
header.Cap = count
header.Len = count
header.Data = uintptr(unsafe.Pointer(rawEntries))
var out []*Info
for _, entry := range entries {
var mountinfo Info
mountinfo.Mountpoint = C.GoString(&entry.f_mntonname[0])
mountinfo.Source = C.GoString(&entry.f_mntfromname[0])
mountinfo.Fstype = C.GoString(&entry.f_fstypename[0])
out = append(out, &mountinfo)
}
return out, nil
}
@@ -0,0 +1,93 @@
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
import (
"bufio"
"fmt"
"io"
"os"
"strings"
)
const (
/* 36 35 98:0 /mnt1 /mnt2 rw,noatime master:1 - ext3 /dev/root rw,errors=continue
(1)(2)(3) (4) (5) (6) (7) (8) (9) (10) (11)
(1) mount ID: unique identifier of the mount (may be reused after umount)
(2) parent ID: ID of parent (or of self for the top of the mount tree)
(3) major:minor: value of st_dev for files on filesystem
(4) root: root of the mount within the filesystem
(5) mount point: mount point relative to the process's root
(6) mount options: per mount options
(7) optional fields: zero or more fields of the form "tag[:value]"
(8) separator: marks the end of the optional fields
(9) filesystem type: name of filesystem of the form "type[.subtype]"
(10) mount source: filesystem specific information or "none"
(11) super options: per super block options*/
mountinfoFormat = "%d %d %d:%d %s %s %s %s"
)
// Parse /proc/self/mountinfo because comparing Dev and ino does not work from
// bind mounts
func parseMountTable() ([]*Info, error) {
f, err := os.Open("/proc/self/mountinfo")
if err != nil {
return nil, err
}
defer f.Close()
return parseInfoFile(f)
}
func parseInfoFile(r io.Reader) ([]*Info, error) {
var (
s = bufio.NewScanner(r)
out = []*Info{}
)
for s.Scan() {
if err := s.Err(); err != nil {
return nil, err
}
var (
p = &Info{}
text = s.Text()
optionalFields string
)
if _, err := fmt.Sscanf(text, mountinfoFormat,
&p.ID, &p.Parent, &p.Major, &p.Minor,
&p.Root, &p.Mountpoint, &p.Opts, &optionalFields); err != nil {
return nil, fmt.Errorf("Scanning '%s' failed: %s", text, err)
}
// Safe as mountinfo encodes mountpoints with spaces as \040.
index := strings.Index(text, " - ")
postSeparatorFields := strings.Fields(text[index+3:])
if len(postSeparatorFields) < 3 {
return nil, fmt.Errorf("Error found less than 3 fields post '-' in %q", text)
}
if optionalFields != "-" {
p.Optional = optionalFields
}
p.Fstype = postSeparatorFields[0]
p.Source = postSeparatorFields[1]
p.VfsOpts = strings.Join(postSeparatorFields[2:], " ")
out = append(out, p)
}
return out, nil
}
// PidMountInfo collects the mounts for a specific process ID. If the process
// ID is unknown, it is better to use `GetMounts` which will inspect
// "/proc/self/mountinfo" instead.
func PidMountInfo(pid int) ([]*Info, error) {
f, err := os.Open(fmt.Sprintf("/proc/%d/mountinfo", pid))
if err != nil {
return nil, err
}
defer f.Close()
return parseInfoFile(f)
}
@@ -0,0 +1,12 @@
// +build !windows,!linux,!freebsd freebsd,!cgo
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
import (
"fmt"
"runtime"
)
func parseMountTable() ([]*Info, error) {
return nil, fmt.Errorf("mount.parseMountTable is not implemented on %s/%s", runtime.GOOS, runtime.GOARCH)
}
@@ -0,0 +1,6 @@
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
func parseMountTable() ([]*Info, error) {
// Do NOT return an error!
return nil, nil
}
@@ -0,0 +1,67 @@
package mount // import "github.com/ory/dockertest/v3/docker/pkg/mount"
// MakeShared ensures a mounted filesystem has the SHARED mount option enabled.
// See the supported options in flags.go for further reference.
func MakeShared(mountPoint string) error {
return ensureMountedAs(mountPoint, "shared")
}
// MakeRShared ensures a mounted filesystem has the RSHARED mount option enabled.
// See the supported options in flags.go for further reference.
func MakeRShared(mountPoint string) error {
return ensureMountedAs(mountPoint, "rshared")
}
// MakePrivate ensures a mounted filesystem has the PRIVATE mount option enabled.
// See the supported options in flags.go for further reference.
func MakePrivate(mountPoint string) error {
return ensureMountedAs(mountPoint, "private")
}
// MakeRPrivate ensures a mounted filesystem has the RPRIVATE mount option
// enabled. See the supported options in flags.go for further reference.
func MakeRPrivate(mountPoint string) error {
return ensureMountedAs(mountPoint, "rprivate")
}
// MakeSlave ensures a mounted filesystem has the SLAVE mount option enabled.
// See the supported options in flags.go for further reference.
func MakeSlave(mountPoint string) error {
return ensureMountedAs(mountPoint, "slave")
}
// MakeRSlave ensures a mounted filesystem has the RSLAVE mount option enabled.
// See the supported options in flags.go for further reference.
func MakeRSlave(mountPoint string) error {
return ensureMountedAs(mountPoint, "rslave")
}
// MakeUnbindable ensures a mounted filesystem has the UNBINDABLE mount option
// enabled. See the supported options in flags.go for further reference.
func MakeUnbindable(mountPoint string) error {
return ensureMountedAs(mountPoint, "unbindable")
}
// MakeRUnbindable ensures a mounted filesystem has the RUNBINDABLE mount
// option enabled. See the supported options in flags.go for further reference.
func MakeRUnbindable(mountPoint string) error {
return ensureMountedAs(mountPoint, "runbindable")
}
func ensureMountedAs(mountPoint, options string) error {
mounted, err := Mounted(mountPoint)
if err != nil {
return err
}
if !mounted {
if err := Mount(mountPoint, mountPoint, "none", "bind,rw"); err != nil {
return err
}
}
if _, err = Mounted(mountPoint); err != nil {
return err
}
return ForceMount("", mountPoint, "none", options)
}
+137
View File
@@ -0,0 +1,137 @@
// Package pools provides a collection of pools which provide various
// data types with buffers. These can be used to lower the number of
// memory allocations and reuse buffers.
//
// New pools should be added to this package to allow them to be
// shared across packages.
//
// Utility functions which operate on pools should be added to this
// package to allow them to be reused.
package pools // import "github.com/ory/dockertest/v3/docker/pkg/pools"
import (
"bufio"
"io"
"sync"
"github.com/ory/dockertest/v3/docker/pkg/ioutils"
)
const buffer32K = 32 * 1024
var (
// BufioReader32KPool is a pool which returns bufio.Reader with a 32K buffer.
BufioReader32KPool = newBufioReaderPoolWithSize(buffer32K)
// BufioWriter32KPool is a pool which returns bufio.Writer with a 32K buffer.
BufioWriter32KPool = newBufioWriterPoolWithSize(buffer32K)
buffer32KPool = newBufferPoolWithSize(buffer32K)
)
// BufioReaderPool is a bufio reader that uses sync.Pool.
type BufioReaderPool struct {
pool sync.Pool
}
// newBufioReaderPoolWithSize is unexported because new pools should be
// added here to be shared where required.
func newBufioReaderPoolWithSize(size int) *BufioReaderPool {
return &BufioReaderPool{
pool: sync.Pool{
New: func() interface{} { return bufio.NewReaderSize(nil, size) },
},
}
}
// Get returns a bufio.Reader which reads from r. The buffer size is that of the pool.
func (bufPool *BufioReaderPool) Get(r io.Reader) *bufio.Reader {
buf := bufPool.pool.Get().(*bufio.Reader)
buf.Reset(r)
return buf
}
// Put puts the bufio.Reader back into the pool.
func (bufPool *BufioReaderPool) Put(b *bufio.Reader) {
b.Reset(nil)
bufPool.pool.Put(b)
}
type bufferPool struct {
pool sync.Pool
}
func newBufferPoolWithSize(size int) *bufferPool {
return &bufferPool{
pool: sync.Pool{
New: func() interface{} { return make([]byte, size) },
},
}
}
func (bp *bufferPool) Get() []byte {
return bp.pool.Get().([]byte)
}
func (bp *bufferPool) Put(b []byte) {
bp.pool.Put(b)
}
// Copy is a convenience wrapper which uses a buffer to avoid allocation in io.Copy.
func Copy(dst io.Writer, src io.Reader) (written int64, err error) {
buf := buffer32KPool.Get()
written, err = io.CopyBuffer(dst, src, buf)
buffer32KPool.Put(buf)
return
}
// NewReadCloserWrapper returns a wrapper which puts the bufio.Reader back
// into the pool and closes the reader if it's an io.ReadCloser.
func (bufPool *BufioReaderPool) NewReadCloserWrapper(buf *bufio.Reader, r io.Reader) io.ReadCloser {
return ioutils.NewReadCloserWrapper(r, func() error {
if readCloser, ok := r.(io.ReadCloser); ok {
readCloser.Close()
}
bufPool.Put(buf)
return nil
})
}
// BufioWriterPool is a bufio writer that uses sync.Pool.
type BufioWriterPool struct {
pool sync.Pool
}
// newBufioWriterPoolWithSize is unexported because new pools should be
// added here to be shared where required.
func newBufioWriterPoolWithSize(size int) *BufioWriterPool {
return &BufioWriterPool{
pool: sync.Pool{
New: func() interface{} { return bufio.NewWriterSize(nil, size) },
},
}
}
// Get returns a bufio.Writer which writes to w. The buffer size is that of the pool.
func (bufPool *BufioWriterPool) Get(w io.Writer) *bufio.Writer {
buf := bufPool.pool.Get().(*bufio.Writer)
buf.Reset(w)
return buf
}
// Put puts the bufio.Writer back into the pool.
func (bufPool *BufioWriterPool) Put(b *bufio.Writer) {
b.Reset(nil)
bufPool.pool.Put(b)
}
// NewWriteCloserWrapper returns a wrapper which puts the bufio.Writer back
// into the pool and closes the writer if it's an io.Writecloser.
func (bufPool *BufioWriterPool) NewWriteCloserWrapper(buf *bufio.Writer, w io.Writer) io.WriteCloser {
return ioutils.NewWriteCloserWrapper(w, func() error {
buf.Flush()
if writeCloser, ok := w.(io.WriteCloser); ok {
writeCloser.Close()
}
bufPool.Put(buf)
return nil
})
}
+190
View File
@@ -0,0 +1,190 @@
package stdcopy // import "github.com/ory/dockertest/v3/docker/pkg/stdcopy"
import (
"bytes"
"encoding/binary"
"errors"
"fmt"
"io"
"sync"
)
// StdType is the type of standard stream
// a writer can multiplex to.
type StdType byte
const (
// Stdin represents standard input stream type.
Stdin StdType = iota
// Stdout represents standard output stream type.
Stdout
// Stderr represents standard error steam type.
Stderr
// Systemerr represents errors originating from the system that make it
// into the the multiplexed stream.
Systemerr
stdWriterPrefixLen = 8
stdWriterFdIndex = 0
stdWriterSizeIndex = 4
startingBufLen = 32*1024 + stdWriterPrefixLen + 1
)
var bufPool = &sync.Pool{New: func() interface{} { return bytes.NewBuffer(nil) }}
// stdWriter is wrapper of io.Writer with extra customized info.
type stdWriter struct {
io.Writer
prefix byte
}
// Write sends the buffer to the underneath writer.
// It inserts the prefix header before the buffer,
// so stdcopy.StdCopy knows where to multiplex the output.
// It makes stdWriter to implement io.Writer.
func (w *stdWriter) Write(p []byte) (n int, err error) {
if w == nil || w.Writer == nil {
return 0, errors.New("Writer not instantiated")
}
if p == nil {
return 0, nil
}
header := [stdWriterPrefixLen]byte{stdWriterFdIndex: w.prefix}
binary.BigEndian.PutUint32(header[stdWriterSizeIndex:], uint32(len(p)))
buf := bufPool.Get().(*bytes.Buffer)
buf.Write(header[:])
buf.Write(p)
n, err = w.Writer.Write(buf.Bytes())
n -= stdWriterPrefixLen
if n < 0 {
n = 0
}
buf.Reset()
bufPool.Put(buf)
return
}
// NewStdWriter instantiates a new Writer.
// Everything written to it will be encapsulated using a custom format,
// and written to the underlying `w` stream.
// This allows multiple write streams (e.g. stdout and stderr) to be muxed into a single connection.
// `t` indicates the id of the stream to encapsulate.
// It can be stdcopy.Stdin, stdcopy.Stdout, stdcopy.Stderr.
func NewStdWriter(w io.Writer, t StdType) io.Writer {
return &stdWriter{
Writer: w,
prefix: byte(t),
}
}
// StdCopy is a modified version of io.Copy.
//
// StdCopy will demultiplex `src`, assuming that it contains two streams,
// previously multiplexed together using a StdWriter instance.
// As it reads from `src`, StdCopy will write to `dstout` and `dsterr`.
//
// StdCopy will read until it hits EOF on `src`. It will then return a nil error.
// In other words: if `err` is non nil, it indicates a real underlying error.
//
// `written` will hold the total number of bytes written to `dstout` and `dsterr`.
func StdCopy(dstout, dsterr io.Writer, src io.Reader) (written int64, err error) {
var (
buf = make([]byte, startingBufLen)
bufLen = len(buf)
nr, nw int
er, ew error
out io.Writer
frameSize int
)
for {
// Make sure we have at least a full header
for nr < stdWriterPrefixLen {
var nr2 int
nr2, er = src.Read(buf[nr:])
nr += nr2
if er == io.EOF {
if nr < stdWriterPrefixLen {
return written, nil
}
break
}
if er != nil {
return 0, er
}
}
stream := StdType(buf[stdWriterFdIndex])
// Check the first byte to know where to write
switch stream {
case Stdin:
fallthrough
case Stdout:
// Write on stdout
out = dstout
case Stderr:
// Write on stderr
out = dsterr
case Systemerr:
// If we're on Systemerr, we won't write anywhere.
// NB: if this code changes later, make sure you don't try to write
// to outstream if Systemerr is the stream
out = nil
default:
return 0, fmt.Errorf("Unrecognized input header: %d", buf[stdWriterFdIndex])
}
// Retrieve the size of the frame
frameSize = int(binary.BigEndian.Uint32(buf[stdWriterSizeIndex : stdWriterSizeIndex+4]))
// Check if the buffer is big enough to read the frame.
// Extend it if necessary.
if frameSize+stdWriterPrefixLen > bufLen {
buf = append(buf, make([]byte, frameSize+stdWriterPrefixLen-bufLen+1)...)
bufLen = len(buf)
}
// While the amount of bytes read is less than the size of the frame + header, we keep reading
for nr < frameSize+stdWriterPrefixLen {
var nr2 int
nr2, er = src.Read(buf[nr:])
nr += nr2
if er == io.EOF {
if nr < frameSize+stdWriterPrefixLen {
return written, nil
}
break
}
if er != nil {
return 0, er
}
}
// we might have an error from the source mixed up in our multiplexed
// stream. if we do, return it.
if stream == Systemerr {
return written, fmt.Errorf("error from daemon in stream: %s", string(buf[stdWriterPrefixLen:frameSize+stdWriterPrefixLen]))
}
// Write the retrieved frame (without header)
nw, ew = out.Write(buf[stdWriterPrefixLen : frameSize+stdWriterPrefixLen])
if ew != nil {
return 0, ew
}
// If the frame has not been fully written: error
if nw != frameSize {
return 0, io.ErrShortWrite
}
written += int64(nw)
// Move the rest of the buffer to the beginning
copy(buf, buf[frameSize+stdWriterPrefixLen:])
// Move the index
nr -= frameSize + stdWriterPrefixLen
}
}
+31
View File
@@ -0,0 +1,31 @@
package system // import "github.com/ory/dockertest/v3/docker/pkg/system"
import (
"os"
"time"
)
// Chtimes changes the access time and modified time of a file at the given path
func Chtimes(name string, atime time.Time, mtime time.Time) error {
unixMinTime := time.Unix(0, 0)
unixMaxTime := maxTime
// If the modified time is prior to the Unix Epoch, or after the
// end of Unix Time, os.Chtimes has undefined behavior
// default to Unix Epoch in this case, just in case
if atime.Before(unixMinTime) || atime.After(unixMaxTime) {
atime = unixMinTime
}
if mtime.Before(unixMinTime) || mtime.After(unixMaxTime) {
mtime = unixMinTime
}
if err := os.Chtimes(name, atime, mtime); err != nil {
return err
}
// Take platform specific action for setting create time.
return setCTime(name, mtime)
}
+14
View File
@@ -0,0 +1,14 @@
// +build !windows
package system // import "github.com/ory/dockertest/v3/docker/pkg/system"
import (
"time"
)
//setCTime will set the create time on a file. On Unix, the create
//time is updated as a side effect of setting the modified time, so
//no action is required.
func setCTime(path string, ctime time.Time) error {
return nil
}
@@ -0,0 +1,26 @@
package system // import "github.com/ory/dockertest/v3/docker/pkg/system"
import (
"time"
"golang.org/x/sys/windows"
)
//setCTime will set the create time on a file. On Windows, this requires
//calling SetFileTime and explicitly including the create time.
func setCTime(path string, ctime time.Time) error {
ctimespec := windows.NsecToTimespec(ctime.UnixNano())
pathp, e := windows.UTF16PtrFromString(path)
if e != nil {
return e
}
h, e := windows.CreateFile(pathp,
windows.FILE_WRITE_ATTRIBUTES, windows.FILE_SHARE_WRITE, nil,
windows.OPEN_EXISTING, windows.FILE_FLAG_BACKUP_SEMANTICS, 0)
if e != nil {
return e
}
defer windows.Close(h)
c := windows.NsecToFiletime(windows.TimespecToNsec(ctimespec))
return windows.SetFileTime(h, &c, nil, nil)
}
+13
View File
@@ -0,0 +1,13 @@
package system // import "github.com/ory/dockertest/v3/docker/pkg/system"
import (
"errors"
)
var (
// ErrNotSupportedPlatform means the platform is not supported.
ErrNotSupportedPlatform = errors.New("platform and architecture is not supported")
// ErrNotSupportedOperatingSystem means the operating system is not supported.
ErrNotSupportedOperatingSystem = errors.New("operating system is not supported")
)
+19
View File
@@ -0,0 +1,19 @@
package system // import "github.com/ory/dockertest/v3/docker/pkg/system"
import (
"fmt"
"os/exec"
"syscall"
)
// GetExitCode returns the ExitStatus of the specified error if its type is
// exec.ExitError, returns 0 and an error otherwise.
func GetExitCode(err error) (int, error) {
exitCode := 0
if exiterr, ok := err.(*exec.ExitError); ok {
if procExit, ok := exiterr.Sys().(syscall.WaitStatus); ok {
return procExit.ExitStatus(), nil
}
}
return exitCode, fmt.Errorf("failed to get exit code")
}
+67
View File
@@ -0,0 +1,67 @@
// +build !windows
package system // import "github.com/ory/dockertest/v3/docker/pkg/system"
import (
"io/ioutil"
"os"
"path/filepath"
)
// MkdirAllWithACL is a wrapper for MkdirAll on unix systems.
func MkdirAllWithACL(path string, perm os.FileMode, sddl string) error {
return MkdirAll(path, perm, sddl)
}
// MkdirAll creates a directory named path along with any necessary parents,
// with permission specified by attribute perm for all dir created.
func MkdirAll(path string, perm os.FileMode, sddl string) error {
return os.MkdirAll(path, perm)
}
// IsAbs is a platform-specific wrapper for filepath.IsAbs.
func IsAbs(path string) bool {
return filepath.IsAbs(path)
}
// The functions below here are wrappers for the equivalents in the os and ioutils packages.
// They are passthrough on Unix platforms, and only relevant on Windows.
// CreateSequential creates the named file with mode 0666 (before umask), truncating
// it if it already exists. If successful, methods on the returned
// File can be used for I/O; the associated file descriptor has mode
// O_RDWR.
// If there is an error, it will be of type *PathError.
func CreateSequential(name string) (*os.File, error) {
return os.Create(name)
}
// OpenSequential opens the named file for reading. If successful, methods on
// the returned file can be used for reading; the associated file
// descriptor has mode O_RDONLY.
// If there is an error, it will be of type *PathError.
func OpenSequential(name string) (*os.File, error) {
return os.Open(name)
}
// OpenFileSequential is the generalized open call; most users will use Open
// or Create instead. It opens the named file with specified flag
// (O_RDONLY etc.) and perm, (0666 etc.) if applicable. If successful,
// methods on the returned File can be used for I/O.
// If there is an error, it will be of type *PathError.
func OpenFileSequential(name string, flag int, perm os.FileMode) (*os.File, error) {
return os.OpenFile(name, flag, perm)
}
// TempFileSequential creates a new temporary file in the directory dir
// with a name beginning with prefix, opens the file for reading
// and writing, and returns the resulting *os.File.
// If dir is the empty string, TempFile uses the default directory
// for temporary files (see os.TempDir).
// Multiple programs calling TempFile simultaneously
// will not choose the same file. The caller can use f.Name()
// to find the pathname of the file. It is the caller's responsibility
// to remove the file when no longer needed.
func TempFileSequential(dir, prefix string) (f *os.File, err error) {
return ioutil.TempFile(dir, prefix)
}
@@ -0,0 +1,295 @@
package system // import "github.com/ory/dockertest/v3/docker/pkg/system"
import (
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
"syscall"
"time"
"unsafe"
"golang.org/x/sys/windows"
)
const (
// SddlAdministratorsLocalSystem is local administrators plus NT AUTHORITY\System
SddlAdministratorsLocalSystem = "D:P(A;OICI;GA;;;BA)(A;OICI;GA;;;SY)"
// SddlNtvmAdministratorsLocalSystem is NT VIRTUAL MACHINE\Virtual Machines plus local administrators plus NT AUTHORITY\System
SddlNtvmAdministratorsLocalSystem = "D:P(A;OICI;GA;;;S-1-5-83-0)(A;OICI;GA;;;BA)(A;OICI;GA;;;SY)"
)
// MkdirAllWithACL is a wrapper for MkdirAll that creates a directory
// with an appropriate SDDL defined ACL.
func MkdirAllWithACL(path string, perm os.FileMode, sddl string) error {
return mkdirall(path, true, sddl)
}
// MkdirAll implementation that is volume path aware for Windows.
func MkdirAll(path string, _ os.FileMode, sddl string) error {
return mkdirall(path, false, sddl)
}
// mkdirall is a custom version of os.MkdirAll modified for use on Windows
// so that it is both volume path aware, and can create a directory with
// a DACL.
func mkdirall(path string, applyACL bool, sddl string) error {
if re := regexp.MustCompile(`^\\\\\?\\Volume{[a-z0-9-]+}$`); re.MatchString(path) {
return nil
}
// The rest of this method is largely copied from os.MkdirAll and should be kept
// as-is to ensure compatibility.
// Fast path: if we can tell whether path is a directory or file, stop with success or error.
dir, err := os.Stat(path)
if err == nil {
if dir.IsDir() {
return nil
}
return &os.PathError{
Op: "mkdir",
Path: path,
Err: syscall.ENOTDIR,
}
}
// Slow path: make sure parent exists and then call Mkdir for path.
i := len(path)
for i > 0 && os.IsPathSeparator(path[i-1]) { // Skip trailing path separator.
i--
}
j := i
for j > 0 && !os.IsPathSeparator(path[j-1]) { // Scan backward over element.
j--
}
if j > 1 {
// Create parent
err = mkdirall(path[0:j-1], false, sddl)
if err != nil {
return err
}
}
// Parent now exists; invoke os.Mkdir or mkdirWithACL and use its result.
if applyACL {
err = mkdirWithACL(path, sddl)
} else {
err = os.Mkdir(path, 0)
}
if err != nil {
// Handle arguments like "foo/." by
// double-checking that directory doesn't exist.
dir, err1 := os.Lstat(path)
if err1 == nil && dir.IsDir() {
return nil
}
return err
}
return nil
}
// mkdirWithACL creates a new directory. If there is an error, it will be of
// type *PathError. .
//
// This is a modified and combined version of os.Mkdir and windows.Mkdir
// in golang to cater for creating a directory am ACL permitting full
// access, with inheritance, to any subfolder/file for Built-in Administrators
// and Local System.
func mkdirWithACL(name string, sddl string) error {
sa := windows.SecurityAttributes{Length: 0}
sd, err := windows.SecurityDescriptorFromString(sddl)
if err != nil {
return &os.PathError{Op: "mkdir", Path: name, Err: err}
}
sa.Length = uint32(unsafe.Sizeof(sa))
sa.InheritHandle = 1
sa.SecurityDescriptor = sd
namep, err := windows.UTF16PtrFromString(name)
if err != nil {
return &os.PathError{Op: "mkdir", Path: name, Err: err}
}
e := windows.CreateDirectory(namep, &sa)
if e != nil {
return &os.PathError{Op: "mkdir", Path: name, Err: e}
}
return nil
}
// IsAbs is a platform-specific wrapper for filepath.IsAbs. On Windows,
// golang filepath.IsAbs does not consider a path \windows\system32 as absolute
// as it doesn't start with a drive-letter/colon combination. However, in
// docker we need to verify things such as WORKDIR /windows/system32 in
// a Dockerfile (which gets translated to \windows\system32 when being processed
// by the daemon. This SHOULD be treated as absolute from a docker processing
// perspective.
func IsAbs(path string) bool {
if !filepath.IsAbs(path) {
if !strings.HasPrefix(path, string(os.PathSeparator)) {
return false
}
}
return true
}
// The origin of the functions below here are the golang OS and windows packages,
// slightly modified to only cope with files, not directories due to the
// specific use case.
//
// The alteration is to allow a file on Windows to be opened with
// FILE_FLAG_SEQUENTIAL_SCAN (particular for docker load), to avoid eating
// the standby list, particularly when accessing large files such as layer.tar.
// CreateSequential creates the named file with mode 0666 (before umask), truncating
// it if it already exists. If successful, methods on the returned
// File can be used for I/O; the associated file descriptor has mode
// O_RDWR.
// If there is an error, it will be of type *PathError.
func CreateSequential(name string) (*os.File, error) {
return OpenFileSequential(name, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0)
}
// OpenSequential opens the named file for reading. If successful, methods on
// the returned file can be used for reading; the associated file
// descriptor has mode O_RDONLY.
// If there is an error, it will be of type *PathError.
func OpenSequential(name string) (*os.File, error) {
return OpenFileSequential(name, os.O_RDONLY, 0)
}
// OpenFileSequential is the generalized open call; most users will use Open
// or Create instead.
// If there is an error, it will be of type *PathError.
func OpenFileSequential(name string, flag int, _ os.FileMode) (*os.File, error) {
if name == "" {
return nil, &os.PathError{Op: "open", Path: name, Err: syscall.ENOENT}
}
r, errf := windowsOpenFileSequential(name, flag, 0)
if errf == nil {
return r, nil
}
return nil, &os.PathError{Op: "open", Path: name, Err: errf}
}
func windowsOpenFileSequential(name string, flag int, _ os.FileMode) (file *os.File, err error) {
r, e := windowsOpenSequential(name, flag|windows.O_CLOEXEC, 0)
if e != nil {
return nil, e
}
return os.NewFile(uintptr(r), name), nil
}
func makeInheritSa() *windows.SecurityAttributes {
var sa windows.SecurityAttributes
sa.Length = uint32(unsafe.Sizeof(sa))
sa.InheritHandle = 1
return &sa
}
func windowsOpenSequential(path string, mode int, _ uint32) (fd windows.Handle, err error) {
if len(path) == 0 {
return windows.InvalidHandle, windows.ERROR_FILE_NOT_FOUND
}
pathp, err := windows.UTF16PtrFromString(path)
if err != nil {
return windows.InvalidHandle, err
}
var access uint32
switch mode & (windows.O_RDONLY | windows.O_WRONLY | windows.O_RDWR) {
case windows.O_RDONLY:
access = windows.GENERIC_READ
case windows.O_WRONLY:
access = windows.GENERIC_WRITE
case windows.O_RDWR:
access = windows.GENERIC_READ | windows.GENERIC_WRITE
}
if mode&windows.O_CREAT != 0 {
access |= windows.GENERIC_WRITE
}
if mode&windows.O_APPEND != 0 {
access &^= windows.GENERIC_WRITE
access |= windows.FILE_APPEND_DATA
}
sharemode := uint32(windows.FILE_SHARE_READ | windows.FILE_SHARE_WRITE)
var sa *windows.SecurityAttributes
if mode&windows.O_CLOEXEC == 0 {
sa = makeInheritSa()
}
var createmode uint32
switch {
case mode&(windows.O_CREAT|windows.O_EXCL) == (windows.O_CREAT | windows.O_EXCL):
createmode = windows.CREATE_NEW
case mode&(windows.O_CREAT|windows.O_TRUNC) == (windows.O_CREAT | windows.O_TRUNC):
createmode = windows.CREATE_ALWAYS
case mode&windows.O_CREAT == windows.O_CREAT:
createmode = windows.OPEN_ALWAYS
case mode&windows.O_TRUNC == windows.O_TRUNC:
createmode = windows.TRUNCATE_EXISTING
default:
createmode = windows.OPEN_EXISTING
}
// Use FILE_FLAG_SEQUENTIAL_SCAN rather than FILE_ATTRIBUTE_NORMAL as implemented in golang.
//https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx
const fileFlagSequentialScan = 0x08000000 // FILE_FLAG_SEQUENTIAL_SCAN
h, e := windows.CreateFile(pathp, access, sharemode, sa, createmode, fileFlagSequentialScan, 0)
return h, e
}
// Helpers for TempFileSequential
var rand uint32
var randmu sync.Mutex
func reseed() uint32 {
return uint32(time.Now().UnixNano() + int64(os.Getpid()))
}
func nextSuffix() string {
randmu.Lock()
r := rand
if r == 0 {
r = reseed()
}
r = r*1664525 + 1013904223 // constants from Numerical Recipes
rand = r
randmu.Unlock()
return strconv.Itoa(int(1e9 + r%1e9))[1:]
}
// TempFileSequential is a copy of ioutil.TempFile, modified to use sequential
// file access. Below is the original comment from golang:
// TempFile creates a new temporary file in the directory dir
// with a name beginning with prefix, opens the file for reading
// and writing, and returns the resulting *os.File.
// If dir is the empty string, TempFile uses the default directory
// for temporary files (see os.TempDir).
// Multiple programs calling TempFile simultaneously
// will not choose the same file. The caller can use f.Name()
// to find the pathname of the file. It is the caller's responsibility
// to remove the file when no longer needed.
func TempFileSequential(dir, prefix string) (f *os.File, err error) {
if dir == "" {
dir = os.TempDir()
}
nconflict := 0
for i := 0; i < 10000; i++ {
name := filepath.Join(dir, prefix+nextSuffix())
f, err = OpenFileSequential(name, os.O_RDWR|os.O_CREATE|os.O_EXCL, 0600)
if os.IsExist(err) {
if nconflict++; nconflict > 10 {
randmu.Lock()
rand = reseed()
randmu.Unlock()
}
continue
}
break
}
return
}
Loaded 100 of 4708 files, more files were not shown because too many files have changed in this diff. Show more