[packman] [opensuse-buildservice] broken builds due to "interconnect error: rpc timeout" with PMBS

Fri Dec 27 19:59:52 CET 2019

Hi Pete,

On Friday, December 27, 2019 2:24:25 PM CET Hans-Peter Jansen wrote:
:
> Interesting. How does your Packman interconnect look like on that
> system?
> 
> E.g. cat /srv/obs/projects/Packman.xml

====
<project name="Packman">
  <title>Packman Build Service PMBS</title>
  <description>This instance can be used to access resources from 
Packman.</description>
  <remoteurl>https://pmbs-api.links2linux.de/public</remoteurl>
</project>
====

> While at it, does systemctl --failed shows any disturbing issues?

No, nothing.

> 
> > I grepped through a month's worth of log files and on both instances
> > there are no mentions of error messages like yours, not even any
> > mention of 'pmbs'.
> > 
> > Did this work before? Up until when?
> 
> Yes, it worked nice for a long time (~08/2017) since about a month
> ago. I first assumed a transient issue, similar to the OBS
> outages/issues in December (which were transient..).
> 
> Later, I updated my obs server with zypper dup from exactly these
> repos:
> 
> # | Alias                     | Name                            |
> Enabled | GPG Check | Refresh
> --+---------------------------+---------------------------------+-----
> ---- +-----------+--------
> 1 | OBS_Server_2.10           | OBS Server 2.10 (openSUSE_15.1) | Yes 
>    | (r ) Yes  | Yes
> 2 | openSUSE:Leap:15.1        | openSUSE:Leap:15.1              | Yes 
>    | (r ) Yes  | Yes
> 3 | openSUSE:Leap:15.1:Update | openSUSE:Leap:15.1:Update       | Yes 
>    | (r ) Yes  | Yes

Yes, I use the same setup.

> Rebooted it several times, etc. All seems well, but still no cigar.
> 
> > PMBS and both my other instances are on 2.10 on 15.1 since July.
> > PMBS and my close instance are on the ESXi behind the reverse proxy
> > since end of August, since then there are just the occasional
> > reboots
> > for security updates.
> 
> I upgraded to 2.10 a little before you (IIRC, since we had similar
> systemd related issues at that time..).
> 
> > I can see at least 8 other IP addresses accessing the public
> > interface today, so to all: Speak up if you too have problems.
> 
> Another interesting observation: it indeed makes a difference, if I
> access the public path of OBS and PMBS with curl or wget.
> 
> (All done from the local obs server):
> 
> $ wget -qO- https://pmbs-api.links2linux.de/public
> <about>
>   <title>Open Build Service API</title>
>   <description>API to the Open Build Service</description>
>   <revision>2.10</revision>
>   <last_deployment>2019-07-05 14:35:37 +0200</last_deployment>
>   <commit>1b66a6c499ef52f61406c333d92731323d852fbc</commit>
> </about>
> 
> $ curl -o- https://pmbs-api.links2linux.de/public
> <html><body>You are being <a href="https://pmbs.links2linux.de/
> about">redirected</a>.</body></html>
> 
> but the behavior is consistent with OBS.

Yes, seems correct, as pmbs.links2linux.{de,org} and pmbs-
api.links2linux.{de,org} are the same machine, the same apache, and 
pmbs.links2linux.de is the primary name (and of course everything is 
reverse proxied through my router VM).

> I can provide many more details, but still have no idea, where these
> issues stem from. It just shows any package build as broken, that
> links to PMBS.

It may be possible, that indeed my setup of PMBS behind a reverse proxy 
is at fault. Is it possible, that access to the weui (html) route works, 
whereas the XML path may be at fault. I am not sure, if there still is a 
difference, and why something has changed, but just hypothetical this 
may be an option to look out for. 
As I am only just a passenger with OBS, too, I am open to suggestions. 
On the other hand I am fine with success stories on access to PMBS ;-)

Greetings,

Stefan
-- 
Stefan Botter zu Hause
Bremen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.links2linux.de/pipermail/packman/attachments/20191227/bd293a75/attachment.sig>