#104 oc tool doesn't work with currently openshift cluster
Opened by bgoncalv. Modified

It seems since the plugins update on Jenkins master the oc tool also got updated, the new version does not work well with the old openshift cluster running jenkins.

sh-4.2$ oc version                                                                                                                                                                                                                 
oc v3.11.0+d545883-301                                                                                                                                                                                                             
kubernetes v1.11.0+d4cacc0                                                                                                                                                                                                         
features: Basic-Auth GSSAPI Kerberos SPNEGO
Server https://172.30.0.1:443                                                                                                                                                                                                      
openshift v3.6.1+008f2d5                                                                                                                                                                                                           
kubernetes v1.6.1+5115d708d7

For example oc get pods fails

sh-4.2$ oc get pods                                                                                                                                                                                                                
No resources found.                                                                                                                                                                                                                
Error from server (NotAcceptable): unknown (get pods)                                                                                                                                                                              
sh-4.2$

This impact our job than clean up old PODs, like https://jenkins-continuous-infra.apps.ci.centos.org/view/all/job/recycle-openshift-builds/663/console

+ oc get pods
+ egrep 'Error|Evicted'
No resources found.
Error from server (NotAcceptable): unknown (get pods)
[Pipeline] sh
++ oc get pods
++ grep ' Completed '
++ awk '{print $1}'
++ grep '\-build'
No resources found.
Error from server (NotAcceptable): unknown (get pods)
+ oc delete pod
error: resource(s) were provided, but no name, label selector, or --all flag specified

@astepano @jimbair could you help investigate this?

@bgoncalv use older oc version, I use oc-3.7 to interact with the cluster, that works ....

I believe there is nothing to fix here, please note that you have oc version 3.11, but the cluster is 3.6

I believe there is nothing to fix here, please note that you have oc version 3.11, but the cluster is 3.6

I can second this. Version mishmash between cli and cluster can lead to really weird errors. Using the same cli version as the cluster is definitely the way to go.

I agree, but why are we using the new version now and what is the best way to install the older one? :)

https://github.com/CentOS-PaaS-SIG/upstream-fedora-pipeline/blob/master/config/s2i/jenkins/slave/Dockerfile#L1

We note this on the slave, but not on the master - do we need a dockerfile for the master?

the jenkins master version seems to be build from https://github.com/CentOS-PaaS-SIG/ci-pipeline/blob/master/config/s2i/jenkins/jenkins-persistent-buildconfig-template.yaml#L78

As we discussed in IRC:

https://github.com/CentOS-PaaS-SIG/ci-pipeline/commit/60dfb9b512e0c47dc876aeed032061d27b9533e0#diff-579465d3c50a19bd07dfe333deee4083R78

This change appears to be what's causing the updated binary

As noted here:
https://hub.docker.com/r/openshift/jenkins-2-centos7/

"This image is now maintained at quay.io/openshift/origin-jenkins"

However, I don't see any 3.x tags in quay, so it's all 4.0+. I assume we would need to update the yaml (maybe only for our instance) to point specifically at 3.6 until we migrate to communishift.

I've pinged @jbieren in IRC (and here now) to see if maybe this shared ci-pipeline repo is being used in other 3.11 clusters, while ours is still on 3.6? Or if we're the only ones using it, maybe we move it back down to 3.6.

As noted here:
https://hub.docker.com/r/openshift/jenkins-2-centos7/
"This image is now maintained at quay.io/openshift/origin-jenkins"
However, I don't see any 3.x tags in quay, so it's all 4.0+. I assume we would need to update the yaml (maybe only for our instance) to point specifically at 3.6 until we migrate to communishift.
I've pinged @jbieren in IRC (and here now) to see if maybe this shared ci-pipeline repo is being used in other 3.11 clusters, while ours is still on 3.6? Or if we're the only ones using it, maybe we move it back down to 3.6.

I don't know for certain of any other projects using the repo, unless the downstream one still is - @bgoncalv is it?

@jbieren do you know which instance the downstream one is? I can take a look from our side (or we can roll back the config and see if we break something else). I also pinged you on IRC. =)

My IRC is on my desktop in the office I am not allowed to go into =\

I don't understand the question of what instance the downstream one is

@jbieren oof - sorry about IRC. =\ Is Google Hangouts good? Email?

As for my question, I was asking as a follow up to your question:

I don't know for certain of any other projects using the repo, unless the downstream one still is - @bgoncalv is it?

My assumption is you mean the RHEL build pipelines?

@jbieren oof - sorry about IRC. =\ Is Google Hangouts good? Email?

Both work great

As for my question, I was asking as a follow up to your question:

I don't know for certain of any other projects using the repo, unless the downstream one still is - @bgoncalv is it?

My assumption is you mean the RHEL build pipelines?

Yeah, I was referring to the rhel8 rpm build pipelines

the rhel8 pipeline doesn't depend on https://github.com/CentOS-PaaS-SIG/ci-pipeline any more.

Proposed fix here:
https://github.com/CentOS-PaaS-SIG/ci-pipeline/pull/829

Metadata