nil pointer dereference when running tests with vault.NewTestCluster on custom Vault plugin











up vote
0
down vote

favorite












I'm developing a custom plugin which is a PKI backend with import to other software functionality (https://github.com/Venafi/vault-pki-monitor-venafi).

This plugin has logic to start import queue in go routine, when role is created, and monitor for new certificates in the role to import them into external system:

- https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L83
- https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_roles.go#L606-L610
To be sure that there is only one import go routine for each role I'm creating a lock path and setting it to true:
https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L88-L153
If lock path doesn't exist I'm assuming it is false:
https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L112-L114



It's all working fine when I'm testing it with vault binary. But when I tried to implement tests using vault.NewTestCluster - https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue_test.go#L16-L63
I got this error:



2018-11-08T15:24:58.004+0300 [DEBUG] core: forwarding: error sending echo request to active node: error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
2018/11/08 15:25:23 Locking import mutex on backend to safely change data for import lock
2018/11/08 15:25:23 Getting import lock for path import-queue-lock/import
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x112863d]


goroutine 557 [running]:
github.com/Venafi/vault-pki-monitor-venafi/plugin/pki.(*backend).importToTPP(0xc4202d3360, 0xc4202ebece, 0x6, 0x14fa8e0, 0xc420040170, 0xc420295a40)
/home/user/src/go/src/github.com/Venafi/vault-pki-monitor-venafi/plugin/pki/path_import_queue.go:105 +0x2fd
created by github.com/Venafi/vault-pki-monitor-venafi/plugin/pki.(*backend).pathRoleCreate
/home/user/src/go/src/github.com/Venafi/vault-pki-monitor-venafi/plugin/pki/path_roles.go:609 +0x2a6b


Here is the place where I'm getting error (this is works fine when running with vault binary, only tests are failing):



log.Printf("Getting import lock for path %s", lockPath)
var importLockEntry *logical.StorageEntry
importLockEntry, err = req.Storage.Get(ctx, lockPath)
if err != nil {
log.Printf("Unable to get lock import for role %s:n %sn", roleName, err)
unlock()
return
}

if importLockEntry == nil || importLockEntry.Value == nil || len(importLockEntry.Value) == 0 {
log.Println("Role lock is empty, assuming it is false")
importLocked = false
} else {
log.Printf("Got from storage %s", string(importLockEntry.Value))
il := string(importLockEntry.Value)
log.Printf("Parsing %s to bool", il)
importLocked, err = strconv.ParseBool(il)
if err != nil {
log.Printf("Unable to parse lock import %s to bool for role %s:n %sn", il, roleName, err)
unlock()
return
}
}


This is how it looks in debugger:
https://13027918075534280191.googlegroups.com/attach/5de3757908e52/Screenshot%20from%202018-11-09%2013-24-52.png



importLockEntry is nil but I'm checking it before usage so it should be fine as I understand. However I'm getting panic right on the https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L105 - the line where I'm trying to set variable.



So, what I'm missing? Why when I run my plugin with vault binary it's working fine and when I'm trying to run i with vault.NewTestCluster it panics?










share|improve this question


















  • 1




    It's not the setting of the variable that panics, it's the reading of req or req.Storage, one of them is nil. Make sure you set up those values properly in your tests.
    – mkopriva
    Nov 9 at 12:43








  • 1




    Yes, you're right, thank you! Will try to setup vault storage before tests.
    – arykalin
    Nov 12 at 9:57















up vote
0
down vote

favorite












I'm developing a custom plugin which is a PKI backend with import to other software functionality (https://github.com/Venafi/vault-pki-monitor-venafi).

This plugin has logic to start import queue in go routine, when role is created, and monitor for new certificates in the role to import them into external system:

- https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L83
- https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_roles.go#L606-L610
To be sure that there is only one import go routine for each role I'm creating a lock path and setting it to true:
https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L88-L153
If lock path doesn't exist I'm assuming it is false:
https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L112-L114



It's all working fine when I'm testing it with vault binary. But when I tried to implement tests using vault.NewTestCluster - https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue_test.go#L16-L63
I got this error:



2018-11-08T15:24:58.004+0300 [DEBUG] core: forwarding: error sending echo request to active node: error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
2018/11/08 15:25:23 Locking import mutex on backend to safely change data for import lock
2018/11/08 15:25:23 Getting import lock for path import-queue-lock/import
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x112863d]


goroutine 557 [running]:
github.com/Venafi/vault-pki-monitor-venafi/plugin/pki.(*backend).importToTPP(0xc4202d3360, 0xc4202ebece, 0x6, 0x14fa8e0, 0xc420040170, 0xc420295a40)
/home/user/src/go/src/github.com/Venafi/vault-pki-monitor-venafi/plugin/pki/path_import_queue.go:105 +0x2fd
created by github.com/Venafi/vault-pki-monitor-venafi/plugin/pki.(*backend).pathRoleCreate
/home/user/src/go/src/github.com/Venafi/vault-pki-monitor-venafi/plugin/pki/path_roles.go:609 +0x2a6b


Here is the place where I'm getting error (this is works fine when running with vault binary, only tests are failing):



log.Printf("Getting import lock for path %s", lockPath)
var importLockEntry *logical.StorageEntry
importLockEntry, err = req.Storage.Get(ctx, lockPath)
if err != nil {
log.Printf("Unable to get lock import for role %s:n %sn", roleName, err)
unlock()
return
}

if importLockEntry == nil || importLockEntry.Value == nil || len(importLockEntry.Value) == 0 {
log.Println("Role lock is empty, assuming it is false")
importLocked = false
} else {
log.Printf("Got from storage %s", string(importLockEntry.Value))
il := string(importLockEntry.Value)
log.Printf("Parsing %s to bool", il)
importLocked, err = strconv.ParseBool(il)
if err != nil {
log.Printf("Unable to parse lock import %s to bool for role %s:n %sn", il, roleName, err)
unlock()
return
}
}


This is how it looks in debugger:
https://13027918075534280191.googlegroups.com/attach/5de3757908e52/Screenshot%20from%202018-11-09%2013-24-52.png



importLockEntry is nil but I'm checking it before usage so it should be fine as I understand. However I'm getting panic right on the https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L105 - the line where I'm trying to set variable.



So, what I'm missing? Why when I run my plugin with vault binary it's working fine and when I'm trying to run i with vault.NewTestCluster it panics?










share|improve this question


















  • 1




    It's not the setting of the variable that panics, it's the reading of req or req.Storage, one of them is nil. Make sure you set up those values properly in your tests.
    – mkopriva
    Nov 9 at 12:43








  • 1




    Yes, you're right, thank you! Will try to setup vault storage before tests.
    – arykalin
    Nov 12 at 9:57













up vote
0
down vote

favorite









up vote
0
down vote

favorite











I'm developing a custom plugin which is a PKI backend with import to other software functionality (https://github.com/Venafi/vault-pki-monitor-venafi).

This plugin has logic to start import queue in go routine, when role is created, and monitor for new certificates in the role to import them into external system:

- https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L83
- https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_roles.go#L606-L610
To be sure that there is only one import go routine for each role I'm creating a lock path and setting it to true:
https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L88-L153
If lock path doesn't exist I'm assuming it is false:
https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L112-L114



It's all working fine when I'm testing it with vault binary. But when I tried to implement tests using vault.NewTestCluster - https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue_test.go#L16-L63
I got this error:



2018-11-08T15:24:58.004+0300 [DEBUG] core: forwarding: error sending echo request to active node: error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
2018/11/08 15:25:23 Locking import mutex on backend to safely change data for import lock
2018/11/08 15:25:23 Getting import lock for path import-queue-lock/import
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x112863d]


goroutine 557 [running]:
github.com/Venafi/vault-pki-monitor-venafi/plugin/pki.(*backend).importToTPP(0xc4202d3360, 0xc4202ebece, 0x6, 0x14fa8e0, 0xc420040170, 0xc420295a40)
/home/user/src/go/src/github.com/Venafi/vault-pki-monitor-venafi/plugin/pki/path_import_queue.go:105 +0x2fd
created by github.com/Venafi/vault-pki-monitor-venafi/plugin/pki.(*backend).pathRoleCreate
/home/user/src/go/src/github.com/Venafi/vault-pki-monitor-venafi/plugin/pki/path_roles.go:609 +0x2a6b


Here is the place where I'm getting error (this is works fine when running with vault binary, only tests are failing):



log.Printf("Getting import lock for path %s", lockPath)
var importLockEntry *logical.StorageEntry
importLockEntry, err = req.Storage.Get(ctx, lockPath)
if err != nil {
log.Printf("Unable to get lock import for role %s:n %sn", roleName, err)
unlock()
return
}

if importLockEntry == nil || importLockEntry.Value == nil || len(importLockEntry.Value) == 0 {
log.Println("Role lock is empty, assuming it is false")
importLocked = false
} else {
log.Printf("Got from storage %s", string(importLockEntry.Value))
il := string(importLockEntry.Value)
log.Printf("Parsing %s to bool", il)
importLocked, err = strconv.ParseBool(il)
if err != nil {
log.Printf("Unable to parse lock import %s to bool for role %s:n %sn", il, roleName, err)
unlock()
return
}
}


This is how it looks in debugger:
https://13027918075534280191.googlegroups.com/attach/5de3757908e52/Screenshot%20from%202018-11-09%2013-24-52.png



importLockEntry is nil but I'm checking it before usage so it should be fine as I understand. However I'm getting panic right on the https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L105 - the line where I'm trying to set variable.



So, what I'm missing? Why when I run my plugin with vault binary it's working fine and when I'm trying to run i with vault.NewTestCluster it panics?










share|improve this question













I'm developing a custom plugin which is a PKI backend with import to other software functionality (https://github.com/Venafi/vault-pki-monitor-venafi).

This plugin has logic to start import queue in go routine, when role is created, and monitor for new certificates in the role to import them into external system:

- https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L83
- https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_roles.go#L606-L610
To be sure that there is only one import go routine for each role I'm creating a lock path and setting it to true:
https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L88-L153
If lock path doesn't exist I'm assuming it is false:
https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L112-L114



It's all working fine when I'm testing it with vault binary. But when I tried to implement tests using vault.NewTestCluster - https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue_test.go#L16-L63
I got this error:



2018-11-08T15:24:58.004+0300 [DEBUG] core: forwarding: error sending echo request to active node: error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
2018/11/08 15:25:23 Locking import mutex on backend to safely change data for import lock
2018/11/08 15:25:23 Getting import lock for path import-queue-lock/import
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x112863d]


goroutine 557 [running]:
github.com/Venafi/vault-pki-monitor-venafi/plugin/pki.(*backend).importToTPP(0xc4202d3360, 0xc4202ebece, 0x6, 0x14fa8e0, 0xc420040170, 0xc420295a40)
/home/user/src/go/src/github.com/Venafi/vault-pki-monitor-venafi/plugin/pki/path_import_queue.go:105 +0x2fd
created by github.com/Venafi/vault-pki-monitor-venafi/plugin/pki.(*backend).pathRoleCreate
/home/user/src/go/src/github.com/Venafi/vault-pki-monitor-venafi/plugin/pki/path_roles.go:609 +0x2a6b


Here is the place where I'm getting error (this is works fine when running with vault binary, only tests are failing):



log.Printf("Getting import lock for path %s", lockPath)
var importLockEntry *logical.StorageEntry
importLockEntry, err = req.Storage.Get(ctx, lockPath)
if err != nil {
log.Printf("Unable to get lock import for role %s:n %sn", roleName, err)
unlock()
return
}

if importLockEntry == nil || importLockEntry.Value == nil || len(importLockEntry.Value) == 0 {
log.Println("Role lock is empty, assuming it is false")
importLocked = false
} else {
log.Printf("Got from storage %s", string(importLockEntry.Value))
il := string(importLockEntry.Value)
log.Printf("Parsing %s to bool", il)
importLocked, err = strconv.ParseBool(il)
if err != nil {
log.Printf("Unable to parse lock import %s to bool for role %s:n %sn", il, roleName, err)
unlock()
return
}
}


This is how it looks in debugger:
https://13027918075534280191.googlegroups.com/attach/5de3757908e52/Screenshot%20from%202018-11-09%2013-24-52.png



importLockEntry is nil but I'm checking it before usage so it should be fine as I understand. However I'm getting panic right on the https://github.com/Venafi/vault-pki-monitor-venafi/blob/adding-tests-for-import/plugin/pki/path_import_queue.go#L105 - the line where I'm trying to set variable.



So, what I'm missing? Why when I run my plugin with vault binary it's working fine and when I'm trying to run i with vault.NewTestCluster it panics?







go hashicorp-vault






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 9 at 12:29









arykalin

9518




9518








  • 1




    It's not the setting of the variable that panics, it's the reading of req or req.Storage, one of them is nil. Make sure you set up those values properly in your tests.
    – mkopriva
    Nov 9 at 12:43








  • 1




    Yes, you're right, thank you! Will try to setup vault storage before tests.
    – arykalin
    Nov 12 at 9:57














  • 1




    It's not the setting of the variable that panics, it's the reading of req or req.Storage, one of them is nil. Make sure you set up those values properly in your tests.
    – mkopriva
    Nov 9 at 12:43








  • 1




    Yes, you're right, thank you! Will try to setup vault storage before tests.
    – arykalin
    Nov 12 at 9:57








1




1




It's not the setting of the variable that panics, it's the reading of req or req.Storage, one of them is nil. Make sure you set up those values properly in your tests.
– mkopriva
Nov 9 at 12:43






It's not the setting of the variable that panics, it's the reading of req or req.Storage, one of them is nil. Make sure you set up those values properly in your tests.
– mkopriva
Nov 9 at 12:43






1




1




Yes, you're right, thank you! Will try to setup vault storage before tests.
– arykalin
Nov 12 at 9:57




Yes, you're right, thank you! Will try to setup vault storage before tests.
– arykalin
Nov 12 at 9:57

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53225741%2fnil-pointer-dereference-when-running-tests-with-vault-newtestcluster-on-custom-v%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes
















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53225741%2fnil-pointer-dereference-when-running-tests-with-vault-newtestcluster-on-custom-v%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







這個網誌中的熱門文章

Academy of Television Arts & Sciences

L'Équipe

1995 France bombings