CloudFlare's WARP Client?

[097115]

It's not possible to use vopono with Cloudflare's WARP Client, I assume? And just in case, no plans to add the support for it? :)

[jamesmcm]

I'll take a look, it really depends how the client works.

I'd never heard of it before though.

[jamesmcm]

Cool, this definitely seems doable btw, the only tricky part will be the killswitch as they set their own firewall rules too.

[jamesmcm]

Basic support is on the master branch now - see PR #237

Note you need to first install and run Warp once:

$ sudo warp-cli register
$ sudo warp-svc  # leave this running
$ sudo warp-cli connect

And then you can kill warp-svc and it should work in vopono:

$ vopono -v exec --no-killswitch --provider warp --protocol warp firefox-developer-edition

Note it doesn't have killswitch support yet, but it should be possible with the information in https://developers.cloudflare.com/cloudflare-one/connections/connect-devices/warp/deployment/firewall/

Also note I can't test the Teams and premium features, so let me know if you're able to try those out.

[097115]

Great news, James, thank you so much!

So, I tried vopono binary from your latest action https://github.com/jamesmcm/vopono/actions/runs/6131725890 like this:

./vopono -v exec --no-killswitch --provider warp --protocol warp $HOME/usr/local/bin/firefox

And got these results:

 2023-09-09T15:57:16.892Z DEBUG vopono_core::util > Using config dir from $HOME config: /home/me/.config
 2023-09-09T15:57:16.909Z DEBUG vopono_core::util::pulseaudio > Setting PULSE_SERVER to /run/user/1000/pulse/native
 2023-09-09T15:57:16.909Z INFO  vopono_core::util             > Calling sudo for elevated privileges, current user will be used as default user
 2023-09-09T15:57:16.909Z DEBUG vopono_core::util             > Args: ["./vopono", "-v", "exec", "--no-killswitch", "--provider", "warp", "--protocol", "warp", "/home/me/usr/local/bin/firefox"]
 2023-09-09T15:57:17.119Z DEBUG vopono_core::util > Using config dir from $HOME config: /home/me/.config
 2023-09-09T15:57:17.135Z DEBUG vopono_core::util::pulseaudio > Setting PULSE_SERVER to /run/user/1000/pulse/native
 2023-09-09T15:57:17.135Z DEBUG vopono_core::util             > Using config dir from $HOME config: /home/me/.config
 2023-09-09T15:57:17.140Z DEBUG vopono_core::util             > Existing namespaces: []
 2023-09-09T15:57:17.140Z DEBUG vopono_core::util             > Using config dir from $HOME config: /home/me/.config
 2023-09-09T15:57:17.141Z DEBUG vopono_core::util             > Using config dir from $HOME config: /home/me/.config
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "firewall" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "custom_config" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "custom_netns_name" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "open_hosts" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "allow_host_access" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "postup" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "predown" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "user" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "group" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "working-directory" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "dns" not found
 2023-09-09T15:57:17.141Z DEBUG vopono::exec                  > vopono config.toml: configuration property "server" not found
thread 'main' panicked at 'Enter a VPN server prefix as a command-line argument or in the vopono config.toml file', src/exec.rs:242:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Seems like it doesn't understand Warp as a VPN provider?

I tried vopono sync, then selecting Warp, but got nothing in my vopono config dir. What am I missing? :)

[jamesmcm]

Oops I didn't see the issue as it was reading some random server name from my vopono config file.

I'll push a fix but for now you can just pass --server warp and it should work.

[097115]

Indeed, it fixes it, thanks again.

However, I'm running into quite a strange situation... So, the details: I have Warp installed, registered, and I have used it before. warp-svc.service is running, and ps returns warp-svc owned by root and warp-taskbar owned be me (just in case, killing any of those makes them restart with the new PIDs).

Now, when I start vopono:

./vopono -v exec --no-killswitch --provider warp --protocol warp $HOME/usr/local/bin/firefox

I get this error in the log:

 2023-09-09T17:14:27.877Z INFO  vopono_core::network::warp              > Launching Warp...
 2023-09-09T17:14:27.877Z DEBUG vopono_core::network::netns             > ip netns exec vo_wp_warp warp-svc
 2023-09-09T17:14:27.878Z DEBUG vopono_core::util                       > Using config dir from $HOME config: /home/me/.config
 2023-09-09T17:14:27.878Z DEBUG vopono_core::network::netns             > Writing lockfile: /home/me/.config/vopono/locks/vo_wp_warp
 2023-09-09T17:14:27.879Z DEBUG vopono_core::network::netns             > Lockfile written: /home/me/.config/vopono/locks/vo_wp_warp/2070
 2023-09-09T17:14:27.879Z DEBUG vopono_core::util                       > Using config dir from $HOME config: /home/me/.config
2023-09-09T17:14:27.915Z  INFO warp::warp_service: Starting WarpService pid=2132
2023-09-09T17:14:27.915Z  INFO warp::warp_service: Version: 2023.7.40
2023-09-09T17:14:27.915Z DEBUG warp_settings::raw_settings: Loading settings from file "/var/lib/cloudflare-warp/settings.json"
2023-09-09T17:14:27.916Z DEBUG warp_settings::manager: Starting local policy file watch parent_path="/var/lib/cloudflare-warp"
 2023-09-09T17:14:27.917Z DEBUG vopono_core::network::netns             > ip netns exec vo_wp_warp sudo --preserve-env --user s /home/me/usr/local/bin/firefox
 2023-09-09T17:14:27.918Z INFO  vopono::exec                            > Application /home/me/usr/local/bin/firefox launched in network namespace vo_wp_warp with pid 2147
2023-09-09T17:14:27.922Z DEBUG watchdog: warp::watchdog: Kicking off watchdog
2023-09-09T17:14:27.922Z ERROR main_loop: warp_net::ipc::unix: Unix socket already bound by root. Is another daemon running? path="/run/cloudflare-warp/warp_service"
2023-09-09T17:14:27.922Z DEBUG main_loop: warp::warp_service: close time.busy=519µs time.idle=15.7µs
2023-09-09T17:14:27.922Z  WARN watchdog: warp::watchdog: Watchdog reports that daemon has disconnected watchdog_name="main loop"
2023-09-09T17:14:27.922Z  INFO warp::warp_service: Dropping WarpService
2023-09-09T17:14:27.922Z DEBUG watchdog: warp::watchdog: close time.busy=749µs time.idle=12.2µs
2023-09-09T17:14:27.922Z  INFO firewall: Firewall stopped
2023-09-09T17:14:27.922Z DEBUG run: warp_settings::manager: close time.busy=37.9µs time.idle=6.64ms
2023-09-09T17:14:27.923Z  INFO warp_svc: Service stopped exit_code=Err(OsError(Custom { kind: Other, error: "Unix socket already bound by root" }))
Error: OsError(Custom { kind: Other, error: "Unix socket already bound by root" })

And firefox simply can't connect :)

You haven't faced any of this?

[jamesmcm]

Don't run warp-svc outside too as it'll interfere with it (as it tries to use the same daemon - see the socket issue there).

So stop / disable the systemd service, kill the warp-svc process and run it only via vopono.

[097115]

Yep, I thought so, too.

But after stopping the service (sudo systemctl stop warp-svc.service, and warp-svc is not running any more), and running vopono, I get this in the logs, and firefox still can't connect :(

[097115]

Aha, and if I run warp-cli connect afterwards, then vopono'ed Firefox does connect, but everything else fails :)

[jamesmcm]

Hmm for me it persisted the state of warp-cli connect so it just automatically connects everytime I run warp-svc

But that is the issue in that log:

2023-09-09T17:29:05.865Z DEBUG main_loop: warp::warp_service::ipc_handlers: Ipc Broadcast ResponseStatus: Disconnected(Manual)

That disconnect was run at some point - maybe it persists the disconnect setting too to not automatically connect at all.

We could make it scan the log and check, and run it if needed - a bit like OpenVPN does - it's a pain if it sets the setting globally though (if you use it outside of vopono).

[097115]

Thanks for the comment! I'm afraid I don't exactly follow, though, so I'll re-describe the situation as I face it:

• I stop warp-svc.service
• I launch vopono'ed Firefox, it does start but fails to connect to the Internet (and the log linked above relates to that launch)
• So, as I've said, Firefox fails to connect but everything else works just fine, I'm not using Warp outside vopono, and can see the IP provided by my ISP at ifconfig.co, for instance
• Now, if I run warp-cli connect in some other terminal, the vopono'ed Firefox will start working (the linked log above doesn't contain anything related to this phase), and ifconfig.co opened in that Firefox will report Cloudflare's IP. But then everything else (like Chrome, my another browser, for instance, which is connected to the Internet directly), everything else will fail :)

I'm on Ubuntu 22.04.03 just in case.

And you are launching your Warp / vopono combo somehow different, as I understand?

[jamesmcm]

Weird, I just ran warp-cli connect outside of vopono, then killed warp-svc (I wasn't running it as a service), and then it just works when I run it anytime - inside of vopono or outside.

Can you try running the command inside the network namespace?

$ ls /etc/netns
$ sudo ip netns exec {netns_name_here} warp-cli connect

[097115]

Soemthing's not good here :)

So, I launch:

./vopono -v exec --no-killswitch --provider warp --protocol warp $HOME/usr/local/bin/kitty

And then ls /etc/netns returns just nothing. Though I can see in the log that vo_wp_warp was created...

To compare, if I launch vopono with openvpn protocol:

./vopono -v exec --no-killswitch --custom ~/path/to/config.ovpn --protocol openvpn $HOME/usr/local/bin/kitty

Then ls /etc/netns indeed returns the correct namespace...

Honestly don't know what to say :)

[jamesmcm]

Hmm the log seems fine - but it still didn't connect that time?

[097115]

It actually did! I dind't do warp-cli connect this time so, probably, it's indeed that persisted state you have mentioned earlier.

But! As soon as the vopono'ed process is launched (be it, for instance, kitty or firefox as in my tests above) the "outer" internet (like my other browser connected directly) stops working. This doesn't happen for you?

And after I close vopono, connectivity in my "outer" browser gets restored...

[jamesmcm]

Hmm can you check the contents of /etc/resolv.conf and /etc/vo_wp_warp/resolv.conf ? Maybe it's just the DNS settings.

Otherwise check for any firewalls set with:

$ sudo iptables -L
$ sudo nft list tables

(outside vopono)

And make sure ufw is disabled just in case.

[097115]

can you check the contents of /etc/resolv.conf

This is probably it.

Before running Warp, my /etc/resolve.conf (which is a symlink to /run/systemd/resolve/stub-resolv.conf) sets the nameserver to 127.0.0.53. And this is the default if systemd-resolved is running, I think.

After I run warp-cli connect, Warp...:

...creates a local DNS proxy on the device and binds it to these IP addresses on port 53 (the port designated for DNS traffic):

IPv4: 127.0.2.2 and 127.0.2.3

IPv6:

macOS and Linux: fd01:db8:1111::2 and fd01:db8:1111::3

Windows: ::ffff:127.0.2.2

...and overwrites the said symlink with an actual file, setting the nameserver to the above values.

If I manually edit this file while Warp is running inside vopono, and add another section for the nameserver, setting it to 1.1.1.1, then the 'outer' internet starts working again (and that's why it seemed like it was toggled on and off simultaneously with vopono launching and exiting).

So, the question probably is, why this doesn't happen with your system? :)

can you check the contents of /etc/vo_wp_warp/resolv.conf

After I launch vopono, I get no /etc/vo_wp_warp/ directory and no /etc/vo_wp_warp/resolv.conf file. Should they have been created? May be it's the same reason ls /etc/netns returns nothing inside vopono for me?

[jamesmcm]

Hmm I hit the same issue now, maybe I just avoided it yesterday due to cached DNS lookups.

I don't use systemd-resolved, but I still hit the issue of warp-svc forcibly overwriting /etc/resolv.conf instead of /etc/netns/vo_wp_warp/resolv.conf like we need it to. Also since the latter file isn't created at the moment (by vopono), it defaults to the system one which is why it works.

It's a tricky issue, the only thing I can think of is using unshare to change the way the filesystem is mounted for the warp-svc process - https://unix.stackexchange.com/questions/237636/is-it-possible-to-fake-a-specific-path-for-a-process

But it's a bit of a pain with the shelling out we have currently, since we're essentially already unsharing for the network namespace itself.